-- Logs begin at Mon 2022-08-29 19:25:30 UTC, end at Mon 2022-08-29 20:31:29 UTC. -- Aug 29 19:58:21.536846 dv-cinder-854261-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 29 19:58:21.551109 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 29 19:58:21.552503 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 29 19:58:21 2022] *** Aug 29 19:58:21.552503 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 29 19:58:21.552503 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 29 19:58:21.552503 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: nodename: dv-cinder-854261-2-iscsi Aug 29 19:58:21.552503 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: machine: x86_64 Aug 29 19:58:21.552503 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: clock source: unix Aug 29 19:58:21.552503 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: pcre jit disabled Aug 29 19:58:21.552503 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: detected number of CPU cores: 4 Aug 29 19:58:21.552503 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: current working directory: / Aug 29 19:58:21.552503 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: detected binary path: /usr/bin/uwsgi-core Aug 29 19:58:21.552503 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: your processes number limit is 47680 Aug 29 19:58:21.553100 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: your memory page size is 4096 bytes Aug 29 19:58:21.553100 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: detected max file descriptor number: 2048 Aug 29 19:58:21.553100 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: lock engine: pthread robust mutexes Aug 29 19:58:21.553100 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: thunder lock: enabled Aug 29 19:58:21.553100 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 29 19:58:21.553100 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 29 19:58:21.593404 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: Python main interpreter initialized at 0x55be1c9949c0 Aug 29 19:58:21.593404 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: python threads support enabled Aug 29 19:58:21.593404 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: your server socket listen backlog is limited to 100 connections Aug 29 19:58:21.593404 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: your mercy for graceful operations on workers is 90 seconds Aug 29 19:58:21.593851 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: mapped 403077 bytes (393 KB) for 2 cores Aug 29 19:58:21.593851 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: *** Operational MODE: preforking *** Aug 29 19:58:21.594136 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: *** uWSGI is running in multiple interpreter mode *** Aug 29 19:58:21.594136 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: spawned uWSGI master process (pid: 81765) Aug 29 19:58:21.594345 dv-cinder-854261-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 29 19:58:21.595475 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: spawned uWSGI worker 1 (pid: 81781, cores: 1) Aug 29 19:58:21.595475 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: spawned uWSGI worker 2 (pid: 81782, cores: 1) Aug 29 19:58:21.595475 dv-cinder-854261-2-iscsi devstack@keystone.service[81765]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 29 19:58:25.989581 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 19:58:25.989581 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: from cryptography.utils import int_from_bytes Aug 29 19:58:25.990432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 19:58:25.990432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: from cryptography.utils import int_from_bytes Aug 29 19:58:25.990573 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 19:58:25.990573 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: from cryptography.utils import int_from_bytes Aug 29 19:58:25.990573 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 19:58:25.990573 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: from cryptography.utils import int_from_bytes Aug 29 19:58:26.041550 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 29 19:58:26.041775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 29 19:58:26.041913 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] command line args: [] {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 29 19:58:26.042078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 29 19:58:26.042191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 29 19:58:26.042372 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ================================================================================ {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 29 19:58:26.042465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 29 19:58:26.042631 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] command line args: [] {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 29 19:58:26.042720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] admin_token = **** {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.042888 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] config_dir = [] {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.043017 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 29 19:58:26.043111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.043177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ================================================================================ {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 29 19:58:26.043242 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] config_source = [] {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.043336 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] admin_token = **** {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.043425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] debug = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.043589 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] config_dir = [] {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.043678 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.043904 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.043968 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] config_source = [] {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.044047 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.044110 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] debug = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.044178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.044344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.044451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.044555 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.044626 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.044734 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.044812 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] list_limit = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.044978 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] log_config_append = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.045081 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.045186 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.045280 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.045379 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] log_dir = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.045478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] list_limit = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.045577 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] log_file = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.045676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] log_config_append = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.045811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.045909 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.046035 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.046148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] log_dir = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.046148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] log_file = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.046360 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.046471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.046581 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.046581 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.046814 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.046930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.047041 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.047202 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.047326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.047439 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.047554 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.047666 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.047777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.047899 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.048020 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.048147 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.048261 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.048361 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.048469 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.048585 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.048707 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.048820 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.048931 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.049048 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.049159 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.049307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.73/identity {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.049442 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.049523 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] publish_errors = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.049645 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.049787 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.049956 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.050061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.050128 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.050370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.050464 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.73/identity {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.050605 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.050683 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] publish_errors = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.050841 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] standard_threads = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.050918 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.051075 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.051173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.051250 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.051321 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.051473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.051560 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.051636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] use_journal = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.051706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.051814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] use_json = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.051948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] standard_threads = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.052026 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] use_stderr = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.052161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] use_syslog = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.052244 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.052371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.052456 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.052588 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.052674 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.052772 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] use_journal = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.052960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.053100 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] use_json = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.053233 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.053379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] use_stderr = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.053496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.053631 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] use_syslog = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.053758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.053832 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 19:58:26.053911 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.053981 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.054049 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.054125 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.054269 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.054346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.054518 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.054594 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.054768 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.054853 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.055020 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.055102 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.055255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.055331 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.055494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.055579 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.055726 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.055807 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.055975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.056091 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.056224 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.056363 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.056478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.056587 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.056706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.056818 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.056958 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.057066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.057207 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.057328 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.057440 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.057604 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.057820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.058022 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.058137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.058279 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.058464 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.058610 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.058735 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.058849 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.058972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.059086 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.059196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.059312 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.059431 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.059549 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.059661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.059771 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.059881 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.059991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.060109 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.060221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.060317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.060432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.060542 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.060658 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.060780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.060901 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.061016 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.061016 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.061228 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.061345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.061469 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.061581 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.061769 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.061886 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.062008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.062127 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.062249 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.062362 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.062480 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.062597 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.062709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.062825 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.062944 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.063061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.063173 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.063283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.063392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.063503 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.063690 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.063805 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.063915 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.064024 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.064133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.064242 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.064359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.064471 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.064582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.064692 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.064801 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.064911 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.065029 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.065141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.065250 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.065360 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.065476 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.065630 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.065829 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.065992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.066168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.066339 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.066468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.066563 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.066666 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.066762 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.066762 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.066964 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.067093 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.067277 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.067400 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.067513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.067628 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.067750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.067877 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.068041 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.068219 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.068397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.068568 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.068728 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.068846 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.068952 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.069069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.069211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.069393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.069518 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.069684 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.069945 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.070141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.070315 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.070464 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.070579 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.070692 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.070811 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.070958 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.071143 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.071336 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.071466 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.071625 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.071804 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.071926 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.072047 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.072169 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.072324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.072999 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.073175 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.073351 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.073519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.073644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.073794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.073953 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.074071 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.074184 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.074184 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.074447 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.074606 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.074746 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.074912 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.075104 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.075104 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.075381 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.075511 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.075511 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.075840 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.075840 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.076127 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.076253 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.076366 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.076479 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.076597 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.076711 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.076831 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.076995 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.077179 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.077365 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.077550 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.077760 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.078055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.078239 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.078422 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.078599 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.078599 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.078881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.079028 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.079157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.079157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.079393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.079511 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.079625 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.079738 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.079859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.079974 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.080086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.080086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.080318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.080318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.080613 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.080732 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.080845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.080958 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.081070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.081182 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.081308 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] auth.external = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.081423 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.081535 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.081649 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.081870 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.081977 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.082107 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.082220 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.082325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] auth.password = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.082430 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.082528 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] auth.token = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.082622 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.082716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.082811 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.082917 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.083020 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] auth.external = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.083122 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.083219 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.083323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.083420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.083524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.083622 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.083717 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.083818 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] auth.password = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.083913 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.084007 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] auth.token = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.084109 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] credential.caching = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.084211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.084307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.084402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.084507 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.084610 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.084716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.084813 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.084907 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.085007 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.085102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.085197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.085299 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.085401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.085497 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.085591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] credential.caching = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.085685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.085809 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.085919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.086029 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.086126 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.086221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.086322 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.086418 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.086522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.086759 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.086874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.086996 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.087116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.087231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.087344 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.087456 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.087570 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] federation.caching = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.087690 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.087805 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.087918 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.088030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.088160 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.088839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.088972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.089088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.089088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.089306 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.089419 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.089531 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.089652 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.089807 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.089922 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.090036 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] federation.caching = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.090134 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.090246 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.090371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.090486 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.090486 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.090689 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.090802 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.090802 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.091023 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] identity.caching = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.091137 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.091259 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.091372 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.091484 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.091673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.091791 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.091944 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.092061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.092174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.092287 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.092414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.092530 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.092649 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] identity.caching = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.092763 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.092878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.092999 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.093121 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.093242 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.093356 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.093474 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.093588 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.093714 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.093714 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.093921 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.094016 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.094110 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.094204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.094299 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.094408 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.094506 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.094601 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.094704 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.094804 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.094900 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.095003 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.095106 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.095201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.095296 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.095390 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.095484 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.095586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.095682 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.095778 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.095872 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.095967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.096061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.096164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.096261 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.096356 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.096356 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.096531 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.096636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.096740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.096835 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.096929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.096929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.097104 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.097199 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.097301 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.097397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.097493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.097587 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.097681 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.097844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.097953 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.098052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.098147 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.098147 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.098322 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.098415 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.098517 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.098614 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.098709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.098826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.098939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.099051 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.099177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.099299 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.099413 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.099525 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.099638 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.099751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.099871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.099986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.100099 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.100213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.100325 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.100438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.100559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.100673 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.100786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.100898 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.101011 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.101123 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.101245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.101359 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.101472 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.101585 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.101697 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.101826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.user = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.101950 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.102102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.102216 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.102330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.102443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.102564 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.102678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.102790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.102903 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.103015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.103128 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.103248 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.103364 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.103477 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.103589 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.103702 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.103864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.user = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.103992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.104108 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.104238 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.104353 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.104466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.104578 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.104699 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.104813 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.104925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.105038 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.105274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.105406 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.105530 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.105646 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.105825 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.105941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.106053 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.106166 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.106291 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.106406 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.106521 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.106634 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.106746 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.106860 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.106999 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.107099 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.107212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.107347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.107494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.107610 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.107732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.107848 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.107961 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.108073 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.108186 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.108299 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.108419 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.108534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.108648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.108803 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.108916 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.108916 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.109140 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.109255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.109255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.109467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.109579 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.109692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.109835 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.109950 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.110063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.110175 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.110288 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.110288 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] resource.caching = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.110507 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.110621 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.110621 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.110834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.110947 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.111060 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.111181 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.111295 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.111408 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.111521 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.111633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.111747 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.111867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.111981 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.112127 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.112243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.112356 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.112469 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.112589 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] role.caching = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.112704 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.112818 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] role.driver = sql {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.112930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.113042 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.113155 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] resource.caching = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.113343 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.113459 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.113572 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.113686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.113830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.113943 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.114068 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.114183 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.114294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.114407 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.114519 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.114630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.114742 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.114856 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.114969 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.115081 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.115193 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.115313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.115427 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.115546 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] role.caching = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.115660 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.115773 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] role.driver = sql {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.115886 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.116006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.116120 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.116233 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.116233 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.116443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.116443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.116662 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.117029 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.117142 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.117254 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.117366 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.117478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.117598 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.117726 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.117841 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.117953 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.118067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.118179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.118308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.118423 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.118542 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.118656 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.118769 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.118881 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.119005 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.119122 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.119241 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.119354 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.119354 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.119616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.119616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.119841 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.119954 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.120065 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.120179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.120339 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.120465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.120580 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.120693 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.120835 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.121077 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.121203 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] token.caching = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.121318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.121430 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.121542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.121655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.121809 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.121941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.122057 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.122204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.122320 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.122433 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.122546 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.122666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.122781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.122894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.123006 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.123117 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.123232 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.123352 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.123467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.123579 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.123692 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.123804 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.123916 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.124037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.124151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.124264 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] token.caching = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.124377 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.124489 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.124601 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.124722 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.124845 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.124974 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.125089 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.125209 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.125324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.125436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] database.connection = **** {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.125549 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.125669 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.125839 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.125954 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.126068 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.126181 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.126293 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.126419 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.126534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.126647 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.126759 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.126872 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.126985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.127105 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.127220 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.127332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.127446 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.127559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.127679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.127794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.127915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.128037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.128152 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.128265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.128379 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.128493 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] database.connection = **** {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.128613 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.128727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.128841 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.128954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.129067 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.129183 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.129304 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.129419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.129534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.129647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.129773 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81782) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 29 19:58:26.129888 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.130048 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.130236 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.130486 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.130728 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.130977 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.131219 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.131473 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.131712 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.131960 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.132195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.132445 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.132686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 19:58:26.132860 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81781) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 29 19:58:26.159935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.160173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.160375 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.162652 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.162889 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.163013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.168879 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.171892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.196949 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.197165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.198644 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.199060 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.253219 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.253386 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.303683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.303908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.304279 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.304518 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.460623 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.460856 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.461079 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.461309 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.461531 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.461775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.461928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.462195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.462425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.462669 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.462893 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.463113 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.486378 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.486538 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.486625 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.486768 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.486855 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.486999 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.487104 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.487224 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.487306 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.487437 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.487519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.487651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.487734 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.487860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.499187 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.499316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.512266 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.512438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.512531 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.512666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.512753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.512891 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.512976 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.513110 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.513192 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.513324 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.513407 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.513536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.513620 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81782) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.513782 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81781) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 19:58:26.531133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.531344 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 19:58:26.531607 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.531694 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.531822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 19:58:26.531906 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 19:58:26.532141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.532223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.532362 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 19:58:26.532446 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.532644 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.532726 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.532897 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.532987 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.533414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.533682 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.543115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.543394 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.543667 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.543820 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.543952 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.544105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.544389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.544679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.550953 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81782) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.551753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81781) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.553858 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81782) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.554151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.554441 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.554728 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.554811 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81781) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.555017 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.555106 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.555284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.555368 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.555560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.555650 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.555837 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.555920 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.556103 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.556191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.556380 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.556484 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.556649 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.556730 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.557252 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.557523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.557828 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.579277 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.580545 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.580740 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81782) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.581028 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.582054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81781) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.582346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.585552 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81782) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.585868 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.586155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.587031 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81781) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.587321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.587611 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.593237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81782) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.593524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.595066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81781) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.595357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.597565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81782) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.597893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.598184 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.598472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.598750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.599480 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81781) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.599775 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.600065 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.600355 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.600659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.600801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.600918 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.601093 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.601181 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.601383 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.601463 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.618012 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.618295 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.619484 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.619774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.622508 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81782) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.623668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81781) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.625591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.626188 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.626934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.627239 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.630518 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81782) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.631446 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81781) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.633652 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81782) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.634631 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81781) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.637532 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.637866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.638156 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.638429 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.638570 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.638717 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.638835 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.639013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.639129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.639306 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.639605 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.639891 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.655673 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81782) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.655965 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.656236 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81781) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.656321 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.656519 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.656602 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.656795 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.657068 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.663398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.663970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.665050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.665509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.666671 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.666954 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.667137 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.667424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.669771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81782) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.670070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.670155 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81781) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.670357 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.670442 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.670645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.670758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.670947 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.671034 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.671338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.682220 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81782) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.682511 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.682610 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81781) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.682797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.682904 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.683087 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.683270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.683364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.683567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.683657 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.683859 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.683943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.684146 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.684450 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.699944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81782) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.700422 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81781) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.702994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81782) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.703352 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81781) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.705906 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.706266 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.707339 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.707703 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.708813 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81782) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.709113 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.709323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81781) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.709408 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.709613 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.709948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.716823 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.716944 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.718317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81782) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.718474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81781) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.721205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.721495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.721594 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.721814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.721902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.722098 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.722182 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.722487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.730566 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81781) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.730668 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81782) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.730861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.730945 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.731146 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.731244 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.738068 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81782) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.738183 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81781) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 19:58:26.738355 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.738442 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.738638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.738728 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.738919 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.739003 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.739203 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.739299 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.739479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.739564 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.739757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.739851 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.740039 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.740123 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.740316 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.740398 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.740588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.740678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.740862 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.740942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.741122 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.741203 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.741387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.741469 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.741661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81782) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.742066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81781) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 19:58:26.777251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55be1c9949c0 pid: 81782 (default app) Aug 29 19:58:26.777777 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55be1c9949c0 pid: 81781 (default app) Aug 29 19:58:26.780714 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2553103-0678-4114-ad1b-f4932888f303 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:26.781001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2553103-0678-4114-ad1b-f4932888f303 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:26.781273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2553103-0678-4114-ad1b-f4932888f303 None None] PATH_INFO: `/v3` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:26.785386 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1/1] 10.222.0.73 () {50 vars in 791 bytes} [Mon Aug 29 19:58:26 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:33.515876 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ddca5-467a-4e94-8e25-3841cdb664e2 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:33.516374 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ddca5-467a-4e94-8e25-3841cdb664e2 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:33.516374 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ddca5-467a-4e94-8e25-3841cdb664e2 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:33.523217 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1/2] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:58:33 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:58:33.529453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198282cd-0605-4971-ae51-fdba8f0dff74 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:33.530055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198282cd-0605-4971-ae51-fdba8f0dff74 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:33.530213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198282cd-0605-4971-ae51-fdba8f0dff74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:33.601279 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: INFO dbcounter [None req-198282cd-0605-4971-ae51-fdba8f0dff74 None None] Registered counter for database keystone Aug 29 19:58:33.621922 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG oslo_db.sqlalchemy.engines [None req-198282cd-0605-4971-ae51-fdba8f0dff74 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81782) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 29 19:58:33.628998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writer thread running {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 29 19:58:33.813323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG passlib.handlers.bcrypt [None req-198282cd-0605-4971-ae51-fdba8f0dff74 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81782) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 29 19:58:33.813618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG passlib.handlers.bcrypt [None req-198282cd-0605-4971-ae51-fdba8f0dff74 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81782) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 29 19:58:33.873790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-198282cd-0605-4971-ae51-fdba8f0dff74 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:58:33.875401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-198282cd-0605-4971-ae51-fdba8f0dff74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:58:34.006266 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 2/3] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:58:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 29 19:58:34.482273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ddca5-467a-4e94-8e25-3841cdb664e2 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:34.554188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: INFO dbcounter [None req-153ddca5-467a-4e94-8e25-3841cdb664e2 None None] Registered counter for database keystone Aug 29 19:58:34.575290 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG oslo_db.sqlalchemy.engines [None req-153ddca5-467a-4e94-8e25-3841cdb664e2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81781) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 29 19:58:34.581837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writer thread running {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 29 19:58:34.702803 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ddca5-467a-4e94-8e25-3841cdb664e2 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:34.707467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d70e1-4d36-46e0-9fc8-6f8e73859726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:34.708107 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d70e1-4d36-46e0-9fc8-6f8e73859726 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:34.708388 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d70e1-4d36-46e0-9fc8-6f8e73859726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:34.708665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d70e1-4d36-46e0-9fc8-6f8e73859726 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:34.717297 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d70e1-4d36-46e0-9fc8-6f8e73859726 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:34.771643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: WARNING keystone.common.rbac_enforcer.enforcer [None req-389d70e1-4d36-46e0-9fc8-6f8e73859726 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 29 19:58:34.771908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d70e1-4d36-46e0-9fc8-6f8e73859726 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:34.782085 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 2/4] 10.222.0.73 () {58 vars in 1324 bytes} [Mon Aug 29 19:58:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:58:34.787996 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198282cd-0605-4971-ae51-fdba8f0dff74 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:34.802567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198282cd-0605-4971-ae51-fdba8f0dff74 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:34.806943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa02dab-6951-445a-bb05-8d9a3a575d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:34.808889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa02dab-6951-445a-bb05-8d9a3a575d44 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:34.809167 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa02dab-6951-445a-bb05-8d9a3a575d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:34.809436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa02dab-6951-445a-bb05-8d9a3a575d44 admin admin] PATH_INFO: `/v3/projects/ee34b4bd3d8043258a1a5d3588fa8f9c` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:34.811045 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa02dab-6951-445a-bb05-8d9a3a575d44 admin admin] RBAC: Authorizing `identity:get_project(project_id=ee34b4bd3d8043258a1a5d3588fa8f9c)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:34.866400 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7aa02dab-6951-445a-bb05-8d9a3a575d44 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 29 19:58:34.866747 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa02dab-6951-445a-bb05-8d9a3a575d44 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:34.872325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 3/5] 10.222.0.73 () {56 vars in 1207 bytes} [Mon Aug 29 19:58:34 2022] GET /identity/v3/projects/ee34b4bd3d8043258a1a5d3588fa8f9c => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:36.795904 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac533d-3ac1-4508-be66-60cf459ee75a None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:36.796479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac533d-3ac1-4508-be66-60cf459ee75a None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:36.796479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac533d-3ac1-4508-be66-60cf459ee75a None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:36.800640 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 3/6] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:58:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:58:36.806273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42e56c-0434-46e7-8ae6-a60088561391 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:36.806707 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42e56c-0434-46e7-8ae6-a60088561391 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:36.806824 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42e56c-0434-46e7-8ae6-a60088561391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:36.828522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-db42e56c-0434-46e7-8ae6-a60088561391 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:58:36.830119 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-db42e56c-0434-46e7-8ae6-a60088561391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:58:36.838190 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 4/7] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:58:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 29 19:58:37.113311 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac533d-3ac1-4508-be66-60cf459ee75a None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:37.126598 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac533d-3ac1-4508-be66-60cf459ee75a None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:37.130854 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7d386-0d9a-4279-8756-fda6f671680d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:37.131453 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7d386-0d9a-4279-8756-fda6f671680d admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:37.131731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7d386-0d9a-4279-8756-fda6f671680d admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:37.131997 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7d386-0d9a-4279-8756-fda6f671680d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:37.140564 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7d386-0d9a-4279-8756-fda6f671680d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:37.142366 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7d386-0d9a-4279-8756-fda6f671680d admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:37.151269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 4/8] 10.222.0.73 () {58 vars in 1324 bytes} [Mon Aug 29 19:58:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:58:37.156943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42e56c-0434-46e7-8ae6-a60088561391 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:37.166241 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42e56c-0434-46e7-8ae6-a60088561391 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:37.171484 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dcd788-9535-4f11-b082-656aee383696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:37.172140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcd788-9535-4f11-b082-656aee383696 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:37.172423 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcd788-9535-4f11-b082-656aee383696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:37.172698 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcd788-9535-4f11-b082-656aee383696 admin admin] PATH_INFO: `/v3/users/7ab6b80a7eaf40e691fcd95e0949b22e` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:37.174077 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dcd788-9535-4f11-b082-656aee383696 admin admin] RBAC: Authorizing `identity:get_user(user_id=7ab6b80a7eaf40e691fcd95e0949b22e)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:37.175918 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dcd788-9535-4f11-b082-656aee383696 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:37.177466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 5/9] 10.222.0.73 () {56 vars in 1198 bytes} [Mon Aug 29 19:58:37 2022] GET /identity/v3/users/7ab6b80a7eaf40e691fcd95e0949b22e => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:39.176600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9268cf-3404-4372-8795-d5e7e65ce6d3 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:39.177139 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9268cf-3404-4372-8795-d5e7e65ce6d3 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:39.177139 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9268cf-3404-4372-8795-d5e7e65ce6d3 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:39.178526 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 5/10] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:58:39.184323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee1473-7aab-4277-a3d0-86834fa26f4b None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:39.184785 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee1473-7aab-4277-a3d0-86834fa26f4b None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:39.184884 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee1473-7aab-4277-a3d0-86834fa26f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:39.205001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-06ee1473-7aab-4277-a3d0-86834fa26f4b None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:58:39.206238 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-06ee1473-7aab-4277-a3d0-86834fa26f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:58:39.224800 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61d93d-23a2-41bc-91fc-a1bc70d9f906 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:39.225084 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61d93d-23a2-41bc-91fc-a1bc70d9f906 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:39.225479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61d93d-23a2-41bc-91fc-a1bc70d9f906 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:39.226881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 6/11] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:58:39.232773 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd4df9-adad-4298-a705-4034bd32f716 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:39.233091 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd4df9-adad-4298-a705-4034bd32f716 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:39.233455 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd4df9-adad-4298-a705-4034bd32f716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:39.278195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG passlib.handlers.bcrypt [None req-29dd4df9-adad-4298-a705-4034bd32f716 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81781) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 29 19:58:39.278589 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG passlib.handlers.bcrypt [None req-29dd4df9-adad-4298-a705-4034bd32f716 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81781) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 29 19:58:39.293191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 6/12] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:58:39.327854 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-29dd4df9-adad-4298-a705-4034bd32f716 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:58:39.329572 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-29dd4df9-adad-4298-a705-4034bd32f716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:58:39.434248 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 7/13] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:58:39.559992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee1473-7aab-4277-a3d0-86834fa26f4b None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:39.585489 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:39.667428 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee1473-7aab-4277-a3d0-86834fa26f4b None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:39.700945 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211fafb4-0ffe-4a6a-90cc-67deb0a71dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:39.701512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fafb4-0ffe-4a6a-90cc-67deb0a71dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:39.701857 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fafb4-0ffe-4a6a-90cc-67deb0a71dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:39.702152 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fafb4-0ffe-4a6a-90cc-67deb0a71dc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:39.706628 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd4df9-adad-4298-a705-4034bd32f716 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:39.714815 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211fafb4-0ffe-4a6a-90cc-67deb0a71dc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:58:39.723820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-211fafb4-0ffe-4a6a-90cc-67deb0a71dc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:58:39.723820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:58:39.723820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:58:39.723820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:58:39.723820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:58:39.723820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:58:39.723820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:58:39.723820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:58:39.723820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:58:39.723820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:58:39.723820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:58:39.723820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:58:39.723820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:58:39.723820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:58:39.723820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:58:39.723820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:58:39.724801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:58:39.724801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:58:39.724801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:58:39.724801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:58:39.724801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:58:39.724801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:58:39.724801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:58:39.724801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:58:39.724801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:58:39.724801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:58:39.724801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:58:39.724801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:58:39.724801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:58:39.724801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:58:39.724801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:58:39.724801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:58:39.725622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:58:39.725622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:58:39.725622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:58:39.725622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:58:39.725622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:58:39.725622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:58:39.725622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:58:39.725622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:58:39.725622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:58:39.725622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:58:39.725622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:58:39.725622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:58:39.725622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 7/14] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 19:58:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:58:39.731076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211fafb4-0ffe-4a6a-90cc-67deb0a71dc7 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:39.811262 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd4df9-adad-4298-a705-4034bd32f716 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:39.813540 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:39.815566 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:39.835811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211fafb4-0ffe-4a6a-90cc-67deb0a71dc7 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:39.852569 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396b4b3-0257-49fe-a911-e5b89bf82a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:39.853444 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396b4b3-0257-49fe-a911-e5b89bf82a45 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:39.853761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396b4b3-0257-49fe-a911-e5b89bf82a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:39.854065 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396b4b3-0257-49fe-a911-e5b89bf82a45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:39.871497 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935c5ad-0b64-43df-9924-592665718264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:39.872104 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935c5ad-0b64-43df-9924-592665718264 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:39.872384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935c5ad-0b64-43df-9924-592665718264 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:39.872659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935c5ad-0b64-43df-9924-592665718264 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:39.873331 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935c5ad-0b64-43df-9924-592665718264 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:58:39.873631 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935c5ad-0b64-43df-9924-592665718264 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:39.875090 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935c5ad-0b64-43df-9924-592665718264 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:39.886937 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 8/15] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:58:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:58:39.892504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935c5ad-0b64-43df-9924-592665718264 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:39.924057 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396b4b3-0257-49fe-a911-e5b89bf82a45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:39.925604 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396b4b3-0257-49fe-a911-e5b89bf82a45 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:39.999782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935c5ad-0b64-43df-9924-592665718264 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:40.027057 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 8/16] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:58:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 19:58:40.032722 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396b4b3-0257-49fe-a911-e5b89bf82a45 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:40.035541 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac58570-948c-40db-b947-89fb10367643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:40.035855 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:40.036321 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac58570-948c-40db-b947-89fb10367643 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:40.036652 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac58570-948c-40db-b947-89fb10367643 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:40.036857 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac58570-948c-40db-b947-89fb10367643 None admin] PATH_INFO: `/v3/users/7ab6b80a7eaf40e691fcd95e0949b22e` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:40.038069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac58570-948c-40db-b947-89fb10367643 None admin] RBAC: Authorizing `identity:get_user(user_id=7ab6b80a7eaf40e691fcd95e0949b22e)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:40.039536 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac58570-948c-40db-b947-89fb10367643 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:40.041107 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 9/17] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:58:39 2022] GET /identity/v3/users/7ab6b80a7eaf40e691fcd95e0949b22e => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:40.046853 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac58570-948c-40db-b947-89fb10367643 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:40.062287 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:40.137038 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396b4b3-0257-49fe-a911-e5b89bf82a45 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:40.149978 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac58570-948c-40db-b947-89fb10367643 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:40.171857 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed9670-22c1-4403-8320-63e09f59c797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:40.172715 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed9670-22c1-4403-8320-63e09f59c797 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:40.172997 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed9670-22c1-4403-8320-63e09f59c797 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:40.173273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed9670-22c1-4403-8320-63e09f59c797 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:40.184037 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7bcb9-d23d-457a-8d61-782568cbf692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:40.184665 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7bcb9-d23d-457a-8d61-782568cbf692 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:40.185183 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7bcb9-d23d-457a-8d61-782568cbf692 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:40.185183 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7bcb9-d23d-457a-8d61-782568cbf692 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:40.186330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7bcb9-d23d-457a-8d61-782568cbf692 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:40.188013 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7bcb9-d23d-457a-8d61-782568cbf692 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:40.189407 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 10/18] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:58:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:58:40.192808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed9670-22c1-4403-8320-63e09f59c797 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:40.194364 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed9670-22c1-4403-8320-63e09f59c797 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:40.195154 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7bcb9-d23d-457a-8d61-782568cbf692 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:40.216651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-ffed9670-22c1-4403-8320-63e09f59c797 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:58:40.216651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:58:40.216651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:58:40.216651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:58:40.216651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:58:40.216651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:58:40.216651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:58:40.216651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:58:40.216651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:58:40.216651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:58:40.216651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:58:40.216651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:58:40.216651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:58:40.216651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:58:40.216651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:58:40.216651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:58:40.218425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:58:40.218425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:58:40.218425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:58:40.218425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:58:40.218425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:58:40.218425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:58:40.218425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:58:40.218425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:58:40.218425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:58:40.218425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:58:40.218425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:58:40.218425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:58:40.218425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:58:40.218425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:58:40.218425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:58:40.218425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:58:40.218425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:58:40.219970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:58:40.219970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:58:40.219970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:58:40.219970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 9/19] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:58:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 19:58:40.224130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed9670-22c1-4403-8320-63e09f59c797 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:40.284334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:40.292660 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:40.313025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7bcb9-d23d-457a-8d61-782568cbf692 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:40.331547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed9670-22c1-4403-8320-63e09f59c797 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:40.346046 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61fe15-29ca-4f70-8e40-877c320abb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:40.346675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61fe15-29ca-4f70-8e40-877c320abb1f None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:40.346951 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61fe15-29ca-4f70-8e40-877c320abb1f None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:40.347219 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61fe15-29ca-4f70-8e40-877c320abb1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:40.347943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61fe15-29ca-4f70-8e40-877c320abb1f None admin] RBAC: Adding query filter params (role.id=296ee2c194824485af71618619f82e09, scope.domain.id=default, user.id=7ab6b80a7eaf40e691fcd95e0949b22e) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:58:40.348241 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61fe15-29ca-4f70-8e40-877c320abb1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:40.349674 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61fe15-29ca-4f70-8e40-877c320abb1f None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:40.356854 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 11/20] 10.222.0.73 () {56 vars in 1322 bytes} [Mon Aug 29 19:58:40 2022] GET /identity/v3/role_assignments?user.id=7ab6b80a7eaf40e691fcd95e0949b22e&scope.domain.id=default&role.id=296ee2c194824485af71618619f82e09 => generated 233 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:58:40.366109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d7f6a-f1fa-481e-9ac8-3f8768415fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:40.366744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d7f6a-f1fa-481e-9ac8-3f8768415fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:40.367023 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d7f6a-f1fa-481e-9ac8-3f8768415fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:40.367295 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d7f6a-f1fa-481e-9ac8-3f8768415fc2 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:40.368228 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d7f6a-f1fa-481e-9ac8-3f8768415fc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:58:40.368531 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d7f6a-f1fa-481e-9ac8-3f8768415fc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:40.370749 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d7f6a-f1fa-481e-9ac8-3f8768415fc2 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:40.391390 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 10/21] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 19:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:58:42.641087 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79147dd5-636c-4b33-bd19-0ca26f13f80c None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:42.642131 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79147dd5-636c-4b33-bd19-0ca26f13f80c None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:42.642613 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79147dd5-636c-4b33-bd19-0ca26f13f80c None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:42.644179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 12/22] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:58:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:58:42.649808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd74972-8013-4fa7-97d3-36851b718866 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:42.650259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd74972-8013-4fa7-97d3-36851b718866 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:42.650429 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd74972-8013-4fa7-97d3-36851b718866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:42.674703 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-edd74972-8013-4fa7-97d3-36851b718866 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:58:42.675918 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-edd74972-8013-4fa7-97d3-36851b718866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:58:42.722521 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55791b36-9b4d-439c-ba2a-4d2408419757 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:42.723152 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55791b36-9b4d-439c-ba2a-4d2408419757 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:42.723719 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55791b36-9b4d-439c-ba2a-4d2408419757 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:42.725323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 13/23] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:58:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:58:42.737837 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f409507-3049-4d4a-b0ac-813bd21ebdb2 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:42.738445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f409507-3049-4d4a-b0ac-813bd21ebdb2 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:42.738933 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f409507-3049-4d4a-b0ac-813bd21ebdb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:42.782851 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-6f409507-3049-4d4a-b0ac-813bd21ebdb2 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:58:42.785372 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 11/24] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:58:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:58:42.787731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-6f409507-3049-4d4a-b0ac-813bd21ebdb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:58:42.863873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 14/25] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:58:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:58:43.089543 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73d9c7-636d-4f96-a6d5-f579fd98d911 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:43.090398 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73d9c7-636d-4f96-a6d5-f579fd98d911 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:43.090876 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73d9c7-636d-4f96-a6d5-f579fd98d911 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:43.092488 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 12/26] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:58:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:58:43.103199 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76534fd1-0371-49df-81e0-978e7eb32742 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:43.103784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76534fd1-0371-49df-81e0-978e7eb32742 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:43.104831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76534fd1-0371-49df-81e0-978e7eb32742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:43.130642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-76534fd1-0371-49df-81e0-978e7eb32742 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:58:43.132144 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-76534fd1-0371-49df-81e0-978e7eb32742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:58:43.232816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 15/27] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:58:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:58:43.271319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095feac-379a-4ba3-b81c-0e6833c5ef3d None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:43.271923 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095feac-379a-4ba3-b81c-0e6833c5ef3d None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:43.272429 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095feac-379a-4ba3-b81c-0e6833c5ef3d None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:43.273980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 13/28] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:58:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:58:43.279618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcb0e1-dafc-4b6c-89f5-c3f75dd1905e None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:43.279906 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcb0e1-dafc-4b6c-89f5-c3f75dd1905e None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:43.280238 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcb0e1-dafc-4b6c-89f5-c3f75dd1905e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:43.297912 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:43.315941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095feac-379a-4ba3-b81c-0e6833c5ef3d None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:43.318313 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-f7fcb0e1-dafc-4b6c-89f5-c3f75dd1905e None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:58:43.319526 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f7fcb0e1-dafc-4b6c-89f5-c3f75dd1905e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:58:43.398584 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:43.416118 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 16/29] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:58:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:58:43.418708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcb0e1-dafc-4b6c-89f5-c3f75dd1905e None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:43.472769 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095feac-379a-4ba3-b81c-0e6833c5ef3d None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:43.549503 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf180137-7136-4994-bcfe-57a79b8b5f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:43.550174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf180137-7136-4994-bcfe-57a79b8b5f35 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:43.550451 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf180137-7136-4994-bcfe-57a79b8b5f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:43.550717 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf180137-7136-4994-bcfe-57a79b8b5f35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:43.572646 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf180137-7136-4994-bcfe-57a79b8b5f35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:58:43.580016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-bf180137-7136-4994-bcfe-57a79b8b5f35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:58:43.580016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:58:43.580016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:58:43.580016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:58:43.580016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:58:43.580016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:58:43.580016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:58:43.580016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:58:43.580016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:58:43.580016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:58:43.580016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:58:43.580016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:58:43.580016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:58:43.580016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:58:43.580016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:58:43.580016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:58:43.581097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:58:43.581097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:58:43.581097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:58:43.581097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:58:43.581097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:58:43.581097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:58:43.581097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:58:43.581097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:58:43.581097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:58:43.581097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:58:43.581097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:58:43.581097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:58:43.581097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:58:43.581097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:58:43.581097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:58:43.581097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:58:43.582046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:58:43.582046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:58:43.582046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:58:43.582046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:58:43.582046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:58:43.582046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:58:43.582046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:58:43.582046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:58:43.582046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:58:43.582046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:58:43.582046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:58:43.582046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:58:43.583806 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 14/30] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 19:58:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 19:58:43.591010 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf180137-7136-4994-bcfe-57a79b8b5f35 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:43.624010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcb0e1-dafc-4b6c-89f5-c3f75dd1905e None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:43.683133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3921ce-b8d9-4316-8d59-e02a3ea9988c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:43.684430 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3921ce-b8d9-4316-8d59-e02a3ea9988c None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:43.684963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3921ce-b8d9-4316-8d59-e02a3ea9988c None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:43.685379 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3921ce-b8d9-4316-8d59-e02a3ea9988c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:43.694745 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:43.704598 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3921ce-b8d9-4316-8d59-e02a3ea9988c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:43.706734 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3921ce-b8d9-4316-8d59-e02a3ea9988c None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:43.736517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-5d3921ce-b8d9-4316-8d59-e02a3ea9988c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:58:43.736517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:58:43.736517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:58:43.736517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:58:43.736517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:58:43.736517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:58:43.736517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:58:43.736517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:58:43.736517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:58:43.736517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:58:43.736517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:58:43.736517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:58:43.736517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:58:43.736517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:58:43.736517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:58:43.736517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:58:43.737700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:58:43.737700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:58:43.737700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:58:43.737700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:58:43.737700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:58:43.737700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:58:43.737700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:58:43.737700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:58:43.737700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:58:43.737700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:58:43.737700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:58:43.737700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:58:43.737700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:58:43.737700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:58:43.737700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:58:43.737700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:58:43.737700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:58:43.738531 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:58:43.738531 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:58:43.738531 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:58:43.740285 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 17/31] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:58:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 19:58:43.746668 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3921ce-b8d9-4316-8d59-e02a3ea9988c None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:43.774133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf180137-7136-4994-bcfe-57a79b8b5f35 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:43.785361 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:43.826751 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2583406-1815-442f-8b90-5c82ba88a98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:43.827407 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2583406-1815-442f-8b90-5c82ba88a98c None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:43.827688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2583406-1815-442f-8b90-5c82ba88a98c None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:43.827961 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2583406-1815-442f-8b90-5c82ba88a98c None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:43.828717 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2583406-1815-442f-8b90-5c82ba88a98c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:58:43.829027 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2583406-1815-442f-8b90-5c82ba88a98c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:43.830679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2583406-1815-442f-8b90-5c82ba88a98c None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:43.845849 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 15/32] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:58:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:43.851166 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2583406-1815-442f-8b90-5c82ba88a98c None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:43.894636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3921ce-b8d9-4316-8d59-e02a3ea9988c None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:43.936534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06bb09-7325-4df8-80ef-bc8c42635de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:43.939340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06bb09-7325-4df8-80ef-bc8c42635de6 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:43.941517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06bb09-7325-4df8-80ef-bc8c42635de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:43.941629 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06bb09-7325-4df8-80ef-bc8c42635de6 None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:43.942346 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06bb09-7325-4df8-80ef-bc8c42635de6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:58:43.942672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06bb09-7325-4df8-80ef-bc8c42635de6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:43.944424 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06bb09-7325-4df8-80ef-bc8c42635de6 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:43.965821 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 18/33] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 19:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:43.971388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06bb09-7325-4df8-80ef-bc8c42635de6 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:43.980555 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2583406-1815-442f-8b90-5c82ba88a98c None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:44.022810 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3178417-6fe5-4037-b35c-d366a87ea597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:44.023908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3178417-6fe5-4037-b35c-d366a87ea597 None admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:44.024389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3178417-6fe5-4037-b35c-d366a87ea597 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:44.024860 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3178417-6fe5-4037-b35c-d366a87ea597 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:44.025744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3178417-6fe5-4037-b35c-d366a87ea597 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:44.028117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:44.028937 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3178417-6fe5-4037-b35c-d366a87ea597 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:44.096696 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06bb09-7325-4df8-80ef-bc8c42635de6 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:44.132322 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf999a8b-2c90-4606-a55a-26e109e19d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:44.132947 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf999a8b-2c90-4606-a55a-26e109e19d40 None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:44.133232 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf999a8b-2c90-4606-a55a-26e109e19d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:44.133510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf999a8b-2c90-4606-a55a-26e109e19d40 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:44.134239 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf999a8b-2c90-4606-a55a-26e109e19d40 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:44.136405 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf999a8b-2c90-4606-a55a-26e109e19d40 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:44.176538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 16/34] 10.222.0.73 () {60 vars in 1131 bytes} [Mon Aug 29 19:58:43 2022] POST /identity/v3/roles => generated 219 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:58:44.177966 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3178417-6fe5-4037-b35c-d366a87ea597 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:44.212512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:44.281070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 19/35] 10.222.0.73 () {60 vars in 1131 bytes} [Mon Aug 29 19:58:43 2022] POST /identity/v3/roles => generated 225 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:58:44.282437 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf999a8b-2c90-4606-a55a-26e109e19d40 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:44.312532 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3178417-6fe5-4037-b35c-d366a87ea597 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:44.384526 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ef24b-8475-4673-a69b-4c6bf70cce83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:44.386138 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ef24b-8475-4673-a69b-4c6bf70cce83 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:44.386573 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ef24b-8475-4673-a69b-4c6bf70cce83 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:44.386991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ef24b-8475-4673-a69b-4c6bf70cce83 None admin] PATH_INFO: `/v3/users/7ab6b80a7eaf40e691fcd95e0949b22e` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:44.389166 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ef24b-8475-4673-a69b-4c6bf70cce83 None admin] RBAC: Authorizing `identity:get_user(user_id=7ab6b80a7eaf40e691fcd95e0949b22e)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:44.391113 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ef24b-8475-4673-a69b-4c6bf70cce83 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:44.393100 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 17/36] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:58:44 2022] GET /identity/v3/users/7ab6b80a7eaf40e691fcd95e0949b22e => generated 247 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:44.394712 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c8b5f-cba4-4f55-bfbe-3bef6c9d1a82 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:44.395160 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c8b5f-cba4-4f55-bfbe-3bef6c9d1a82 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:44.395585 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c8b5f-cba4-4f55-bfbe-3bef6c9d1a82 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:44.397555 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 18/37] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:58:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:58:44.403551 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c7e55-1881-4da4-9bd2-8d4bf636b0ac None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:44.404056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c7e55-1881-4da4-9bd2-8d4bf636b0ac None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:44.404471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c7e55-1881-4da4-9bd2-8d4bf636b0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:44.426391 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-7a8c7e55-1881-4da4-9bd2-8d4bf636b0ac None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:58:44.428803 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-7a8c7e55-1881-4da4-9bd2-8d4bf636b0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:58:44.530829 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 19/38] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:58:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:58:44.532007 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c7e55-1881-4da4-9bd2-8d4bf636b0ac None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:44.537955 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf999a8b-2c90-4606-a55a-26e109e19d40 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:44.548765 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:44.569034 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:44.586687 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d00940-2b71-4dbe-95bf-3684fb5f7791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:44.587749 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d00940-2b71-4dbe-95bf-3684fb5f7791 None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:44.588042 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d00940-2b71-4dbe-95bf-3684fb5f7791 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:44.588487 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d00940-2b71-4dbe-95bf-3684fb5f7791 None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:44.589138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d00940-2b71-4dbe-95bf-3684fb5f7791 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:44.591567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d00940-2b71-4dbe-95bf-3684fb5f7791 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:44.624888 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 20/39] 10.222.0.73 () {60 vars in 1141 bytes} [Mon Aug 29 19:58:44 2022] POST /identity/v3/projects => generated 301 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:58:44.626248 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829262b7-cfbd-438d-ace6-6d89f3adf110 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:44.626761 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829262b7-cfbd-438d-ace6-6d89f3adf110 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:44.627181 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829262b7-cfbd-438d-ace6-6d89f3adf110 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:44.628598 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 21/40] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:58:44.634763 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eca617-f5a0-45ff-9e8e-5957a8fa2711 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:44.635082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eca617-f5a0-45ff-9e8e-5957a8fa2711 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:44.635318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eca617-f5a0-45ff-9e8e-5957a8fa2711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:44.658826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-25eca617-f5a0-45ff-9e8e-5957a8fa2711 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:58:44.660701 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-25eca617-f5a0-45ff-9e8e-5957a8fa2711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:58:44.671129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c7e55-1881-4da4-9bd2-8d4bf636b0ac None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:44.712586 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4441813-d200-4f17-ace0-4c38f8be9e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:44.713471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4441813-d200-4f17-ace0-4c38f8be9e34 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:44.713945 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4441813-d200-4f17-ace0-4c38f8be9e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:44.714369 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4441813-d200-4f17-ace0-4c38f8be9e34 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:44.715856 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4441813-d200-4f17-ace0-4c38f8be9e34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:44.717577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4441813-d200-4f17-ace0-4c38f8be9e34 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:44.724289 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 20/41] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:58:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:44.725597 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4441813-d200-4f17-ace0-4c38f8be9e34 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:44.738638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 22/42] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:58:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:58:44.814634 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eca617-f5a0-45ff-9e8e-5957a8fa2711 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:44.845060 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:44.854790 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4441813-d200-4f17-ace0-4c38f8be9e34 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:44.902004 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ac62e-8382-4a4e-bd68-82ef2f845be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:44.902691 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ac62e-8382-4a4e-bd68-82ef2f845be4 None admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:44.903140 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ac62e-8382-4a4e-bd68-82ef2f845be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:44.903425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ac62e-8382-4a4e-bd68-82ef2f845be4 None admin] PATH_INFO: `/v3/domains/default/users/7ab6b80a7eaf40e691fcd95e0949b22e/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:44.905949 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ac62e-8382-4a4e-bd68-82ef2f845be4 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7ab6b80a7eaf40e691fcd95e0949b22e, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:44.907406 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ac62e-8382-4a4e-bd68-82ef2f845be4 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:44.923554 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 21/43] 10.222.0.73 () {58 vars in 1361 bytes} [Mon Aug 29 19:58:44 2022] PUT /identity/v3/domains/default/users/7ab6b80a7eaf40e691fcd95e0949b22e/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:58:44.930663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:44.950236 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eca617-f5a0-45ff-9e8e-5957a8fa2711 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:44.992763 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a7a6d-69d4-4a08-8693-f47c099daa0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:44.993724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a7a6d-69d4-4a08-8693-f47c099daa0e None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:44.994087 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a7a6d-69d4-4a08-8693-f47c099daa0e None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:44.994365 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a7a6d-69d4-4a08-8693-f47c099daa0e None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:44.995813 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a7a6d-69d4-4a08-8693-f47c099daa0e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:44.997368 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a7a6d-69d4-4a08-8693-f47c099daa0e None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:45.017743 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 23/44] 10.222.0.73 () {60 vars in 1131 bytes} [Mon Aug 29 19:58:44 2022] POST /identity/v3/roles => generated 223 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:58:45.043267 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ac62e-8382-4a4e-bd68-82ef2f845be4 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:45.177970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ac62e-8382-4a4e-bd68-82ef2f845be4 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:45.215721 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371849d6-1bf8-4cd4-a826-457656077732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:45.216414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371849d6-1bf8-4cd4-a826-457656077732 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:45.216697 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371849d6-1bf8-4cd4-a826-457656077732 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:45.216971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371849d6-1bf8-4cd4-a826-457656077732 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:45.218128 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371849d6-1bf8-4cd4-a826-457656077732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:45.219755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371849d6-1bf8-4cd4-a826-457656077732 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:45.221486 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 22/45] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:58:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:45.227140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a7a6d-69d4-4a08-8693-f47c099daa0e None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:45.337954 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a7a6d-69d4-4a08-8693-f47c099daa0e None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:45.375542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1ae16-b612-4aa4-bc71-a47ef3bbcaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:45.376076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1ae16-b612-4aa4-bc71-a47ef3bbcaed None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:45.376356 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1ae16-b612-4aa4-bc71-a47ef3bbcaed None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:45.376627 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1ae16-b612-4aa4-bc71-a47ef3bbcaed None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:45.377294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1ae16-b612-4aa4-bc71-a47ef3bbcaed None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:45.378771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1ae16-b612-4aa4-bc71-a47ef3bbcaed None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:45.401749 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:45.410015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 24/46] 10.222.0.73 () {60 vars in 1141 bytes} [Mon Aug 29 19:58:45 2022] POST /identity/v3/projects => generated 312 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:58:46.867716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6f760-d197-4478-9e1a-75d1e5ee98d8 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:46.868526 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6f760-d197-4478-9e1a-75d1e5ee98d8 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:46.868614 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6f760-d197-4478-9e1a-75d1e5ee98d8 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:46.870158 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 23/47] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:58:46.875970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2dc99-7f0c-4dd4-9c2e-dfef36db2524 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:46.876426 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2dc99-7f0c-4dd4-9c2e-dfef36db2524 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:46.876531 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2dc99-7f0c-4dd4-9c2e-dfef36db2524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:46.895956 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-73c2dc99-7f0c-4dd4-9c2e-dfef36db2524 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:58:46.897142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-73c2dc99-7f0c-4dd4-9c2e-dfef36db2524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:58:46.970411 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 25/48] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:58:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:58:47.246349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e6f760-d197-4478-9e1a-75d1e5ee98d8 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:47.272423 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:47.326737 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d252daf-238d-41ed-9da3-97ab4e0fdff2 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:47.327017 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d252daf-238d-41ed-9da3-97ab4e0fdff2 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:47.327288 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d252daf-238d-41ed-9da3-97ab4e0fdff2 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:47.328615 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 26/49] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:58:47.333517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b166c-4b12-48ea-9304-8e80b15431b5 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:47.333874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b166c-4b12-48ea-9304-8e80b15431b5 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:47.334122 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b166c-4b12-48ea-9304-8e80b15431b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:47.352321 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-136b166c-4b12-48ea-9304-8e80b15431b5 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:58:47.353513 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-136b166c-4b12-48ea-9304-8e80b15431b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:58:47.355683 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e6f760-d197-4478-9e1a-75d1e5ee98d8 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:47.390176 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b8ae3-a6f9-4d7c-9331-d05793411abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:47.390817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b8ae3-a6f9-4d7c-9331-d05793411abc None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:47.391096 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b8ae3-a6f9-4d7c-9331-d05793411abc None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:47.391369 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b8ae3-a6f9-4d7c-9331-d05793411abc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:47.402068 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b8ae3-a6f9-4d7c-9331-d05793411abc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:58:47.407424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-b80b8ae3-a6f9-4d7c-9331-d05793411abc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:58:47.407424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:58:47.407424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:58:47.407424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:58:47.407424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:58:47.407424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:58:47.407424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:58:47.407424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:58:47.407424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:58:47.407424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:58:47.407424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:58:47.407424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:58:47.407424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:58:47.407424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:58:47.407424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:58:47.407424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:58:47.408448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:58:47.408448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:58:47.408448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:58:47.408448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:58:47.408448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:58:47.408448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:58:47.408448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:58:47.408448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:58:47.408448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:58:47.408448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:58:47.408448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:58:47.408448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:58:47.408448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:58:47.408448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:58:47.408448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:58:47.408448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:58:47.409215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:58:47.409215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:58:47.409215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:58:47.409215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:58:47.409215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:58:47.409215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:58:47.409215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:58:47.409215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:58:47.409215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:58:47.409215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:58:47.409215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:58:47.409215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:58:47.409215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 24/50] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 19:58:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:58:47.414281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b8ae3-a6f9-4d7c-9331-d05793411abc None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:47.421675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 27/51] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:58:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:58:47.496086 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:47.514994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b8ae3-a6f9-4d7c-9331-d05793411abc None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:47.548418 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9445985a-d970-45f9-bc81-1c40a9eeaf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:47.549029 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445985a-d970-45f9-bc81-1c40a9eeaf1c None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:47.549309 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445985a-d970-45f9-bc81-1c40a9eeaf1c None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:47.549581 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445985a-d970-45f9-bc81-1c40a9eeaf1c None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:47.550281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9445985a-d970-45f9-bc81-1c40a9eeaf1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:58:47.550586 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9445985a-d970-45f9-bc81-1c40a9eeaf1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:47.551961 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9445985a-d970-45f9-bc81-1c40a9eeaf1c None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:47.561006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 25/52] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:58:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:58:47.566593 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b166c-4b12-48ea-9304-8e80b15431b5 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:47.624900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:47.665183 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b166c-4b12-48ea-9304-8e80b15431b5 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:47.691920 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9445985a-d970-45f9-bc81-1c40a9eeaf1c None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:47.699455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df1df6-2bb6-47de-bbec-1333c369e2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:47.700096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df1df6-2bb6-47de-bbec-1333c369e2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:47.700376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df1df6-2bb6-47de-bbec-1333c369e2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:47.700649 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df1df6-2bb6-47de-bbec-1333c369e2c1 None admin] PATH_INFO: `/v3/users/7ab6b80a7eaf40e691fcd95e0949b22e` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:47.701810 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65df1df6-2bb6-47de-bbec-1333c369e2c1 None admin] RBAC: Authorizing `identity:get_user(user_id=7ab6b80a7eaf40e691fcd95e0949b22e)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:47.703271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65df1df6-2bb6-47de-bbec-1333c369e2c1 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:47.705000 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 28/53] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:58:47 2022] GET /identity/v3/users/7ab6b80a7eaf40e691fcd95e0949b22e => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:47.710369 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df1df6-2bb6-47de-bbec-1333c369e2c1 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:47.795200 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9445985a-d970-45f9-bc81-1c40a9eeaf1c None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:47.812957 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df1df6-2bb6-47de-bbec-1333c369e2c1 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:47.832898 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce50caf-9d0e-4dbc-b2dc-11b48449457f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:47.834046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce50caf-9d0e-4dbc-b2dc-11b48449457f None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:47.834553 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce50caf-9d0e-4dbc-b2dc-11b48449457f None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:47.835051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce50caf-9d0e-4dbc-b2dc-11b48449457f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:47.837056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce50caf-9d0e-4dbc-b2dc-11b48449457f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:47.839871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce50caf-9d0e-4dbc-b2dc-11b48449457f None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:47.840777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:47.845750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 26/54] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:58:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:58:47.849592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce50caf-9d0e-4dbc-b2dc-11b48449457f None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:47.850950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c580f58-e32c-4cf9-9675-afe8a42c7046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:47.851608 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c580f58-e32c-4cf9-9675-afe8a42c7046 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:47.851893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c580f58-e32c-4cf9-9675-afe8a42c7046 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:47.852173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c580f58-e32c-4cf9-9675-afe8a42c7046 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:47.854193 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c580f58-e32c-4cf9-9675-afe8a42c7046 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:47.856241 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c580f58-e32c-4cf9-9675-afe8a42c7046 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:47.857894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 29/55] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:58:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:58:47.863742 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c580f58-e32c-4cf9-9675-afe8a42c7046 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:47.865581 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:47.954204 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce50caf-9d0e-4dbc-b2dc-11b48449457f None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:47.965587 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c580f58-e32c-4cf9-9675-afe8a42c7046 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:48.004595 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98979066-f70a-4c78-8cb8-1c042e5107d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:48.005230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98979066-f70a-4c78-8cb8-1c042e5107d7 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:48.005515 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98979066-f70a-4c78-8cb8-1c042e5107d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:48.005823 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98979066-f70a-4c78-8cb8-1c042e5107d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:48.006586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98979066-f70a-4c78-8cb8-1c042e5107d7 None admin] RBAC: Adding query filter params (role.id=296ee2c194824485af71618619f82e09, scope.domain.id=default, user.id=7ab6b80a7eaf40e691fcd95e0949b22e) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:58:48.006899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98979066-f70a-4c78-8cb8-1c042e5107d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:48.008392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98979066-f70a-4c78-8cb8-1c042e5107d7 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:48.015188 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 30/56] 10.222.0.73 () {56 vars in 1322 bytes} [Mon Aug 29 19:58:47 2022] GET /identity/v3/role_assignments?user.id=7ab6b80a7eaf40e691fcd95e0949b22e&scope.domain.id=default&role.id=296ee2c194824485af71618619f82e09 => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:58:48.102382 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7db2f-204f-4c47-aeca-3b90f489b559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:48.103007 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7db2f-204f-4c47-aeca-3b90f489b559 None admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:48.103287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7db2f-204f-4c47-aeca-3b90f489b559 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:48.103559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7db2f-204f-4c47-aeca-3b90f489b559 None admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:48.104288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7db2f-204f-4c47-aeca-3b90f489b559 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:48.107161 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7db2f-204f-4c47-aeca-3b90f489b559 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:48.139734 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 27/57] 10.222.0.73 () {60 vars in 1140 bytes} [Mon Aug 29 19:58:47 2022] POST /identity/v3/projects => generated 298 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:58:50.065990 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432c6d9-cb5e-4196-b792-64e884c1cf81 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:50.066511 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432c6d9-cb5e-4196-b792-64e884c1cf81 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:50.066597 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432c6d9-cb5e-4196-b792-64e884c1cf81 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:50.067900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 31/58] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:58:50.073681 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac845744-2012-44f1-87ef-2122cd3d9455 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:50.074358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac845744-2012-44f1-87ef-2122cd3d9455 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:50.074358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac845744-2012-44f1-87ef-2122cd3d9455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:50.093533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-ac845744-2012-44f1-87ef-2122cd3d9455 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:58:50.094803 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-ac845744-2012-44f1-87ef-2122cd3d9455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:58:50.152913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:50.161984 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 28/59] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:58:50.434824 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432c6d9-cb5e-4196-b792-64e884c1cf81 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:50.495969 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:50.539293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432c6d9-cb5e-4196-b792-64e884c1cf81 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:50.574508 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84924c27-4722-448c-8292-3c0559c45349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:50.575131 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84924c27-4722-448c-8292-3c0559c45349 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:50.575414 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84924c27-4722-448c-8292-3c0559c45349 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:50.575688 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84924c27-4722-448c-8292-3c0559c45349 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:50.576809 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84924c27-4722-448c-8292-3c0559c45349 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:50.578304 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84924c27-4722-448c-8292-3c0559c45349 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:50.579873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 32/60] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:58:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:50.585659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac845744-2012-44f1-87ef-2122cd3d9455 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:50.684712 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac845744-2012-44f1-87ef-2122cd3d9455 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:50.717648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce182050-e32a-40fc-9a4d-9eec03e2b82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:50.718223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce182050-e32a-40fc-9a4d-9eec03e2b82c None admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:50.718508 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce182050-e32a-40fc-9a4d-9eec03e2b82c None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:50.718785 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce182050-e32a-40fc-9a4d-9eec03e2b82c None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:50.719441 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce182050-e32a-40fc-9a4d-9eec03e2b82c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:50.720892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce182050-e32a-40fc-9a4d-9eec03e2b82c None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:50.748800 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 29/61] 10.222.0.73 () {60 vars in 1132 bytes} [Mon Aug 29 19:58:50 2022] POST /identity/v3/users => generated 275 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:58:53.741427 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc2cf6-4281-4d18-9c39-9358b34f8399 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:53.742634 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc2cf6-4281-4d18-9c39-9358b34f8399 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:53.743214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc2cf6-4281-4d18-9c39-9358b34f8399 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:53.745802 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 33/62] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:58:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:58:53.751436 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceb6f9-f176-4dd3-bb20-907120d6d018 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:53.752263 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceb6f9-f176-4dd3-bb20-907120d6d018 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:53.752769 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceb6f9-f176-4dd3-bb20-907120d6d018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:53.774780 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-78ceb6f9-f176-4dd3-bb20-907120d6d018 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:58:53.776419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-78ceb6f9-f176-4dd3-bb20-907120d6d018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:58:53.790305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4485e-fa44-4c3d-9b67-494e5d1d7f87 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:53.790907 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4485e-fa44-4c3d-9b67-494e5d1d7f87 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:53.791392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4485e-fa44-4c3d-9b67-494e5d1d7f87 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:53.793602 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 34/63] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:58:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:58:53.799812 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8e08d-eebe-45a4-9ea8-d12467e5b86a None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:53.800100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8e08d-eebe-45a4-9ea8-d12467e5b86a None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:53.800387 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8e08d-eebe-45a4-9ea8-d12467e5b86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:53.839020 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-d3d8e08d-eebe-45a4-9ea8-d12467e5b86a None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:58:53.840262 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-d3d8e08d-eebe-45a4-9ea8-d12467e5b86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:58:53.844853 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:53.892213 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 30/64] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:58:53.964556 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 35/65] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:58:54.187426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ceb6f9-f176-4dd3-bb20-907120d6d018 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:54.309834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ceb6f9-f176-4dd3-bb20-907120d6d018 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:54.328049 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8e08d-eebe-45a4-9ea8-d12467e5b86a None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:54.356638 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20784fb0-2970-495b-9692-c28e9540c706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:54.357277 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20784fb0-2970-495b-9692-c28e9540c706 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:54.357561 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20784fb0-2970-495b-9692-c28e9540c706 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:54.358101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20784fb0-2970-495b-9692-c28e9540c706 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:54.372184 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20784fb0-2970-495b-9692-c28e9540c706 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:58:54.378838 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-20784fb0-2970-495b-9692-c28e9540c706 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:58:54.378838 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:58:54.378838 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:58:54.378838 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:58:54.378838 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:58:54.378838 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:58:54.378838 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:58:54.378838 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:58:54.378838 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:58:54.378838 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:58:54.378838 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:58:54.378838 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:58:54.378838 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:58:54.378838 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:58:54.378838 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:58:54.378838 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:58:54.379817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:58:54.379817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:58:54.379817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:58:54.379817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:58:54.379817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:58:54.379817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:58:54.379817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:58:54.379817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:58:54.379817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:58:54.379817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:58:54.379817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:58:54.379817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:58:54.379817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:58:54.379817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:58:54.379817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:58:54.379817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:58:54.380559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:58:54.380559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:58:54.380559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:58:54.380559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:58:54.380559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:58:54.380559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:58:54.380559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:58:54.380559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:58:54.380559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:58:54.380559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:58:54.380559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:58:54.380559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:58:54.381192 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 31/66] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:58:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:58:54.383098 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309f54d-7fae-49a9-bb20-28dd0a792218 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:54.383511 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309f54d-7fae-49a9-bb20-28dd0a792218 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:54.383907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309f54d-7fae-49a9-bb20-28dd0a792218 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:54.385337 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 32/67] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:58:54.395133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309f54d-7fae-49a9-bb20-28dd0a792218 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:54.415061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:54.452149 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:54.466266 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8e08d-eebe-45a4-9ea8-d12467e5b86a None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:54.505187 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0ebac-563f-40a4-aad7-33c73b5224f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:54.506252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0ebac-563f-40a4-aad7-33c73b5224f3 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:54.506542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0ebac-563f-40a4-aad7-33c73b5224f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:54.506888 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0ebac-563f-40a4-aad7-33c73b5224f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:54.518109 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0ebac-563f-40a4-aad7-33c73b5224f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:58:54.519247 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309f54d-7fae-49a9-bb20-28dd0a792218 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:54.523342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-b9e0ebac-563f-40a4-aad7-33c73b5224f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:58:54.523342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:58:54.523342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:58:54.523342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:58:54.523342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:58:54.523342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:58:54.523342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:58:54.523342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:58:54.523342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:58:54.523342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:58:54.523342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:58:54.523342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:58:54.523342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:58:54.523342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:58:54.523342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:58:54.523342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:58:54.524216 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:58:54.524216 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:58:54.524216 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:58:54.524216 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:58:54.524216 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:58:54.524216 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:58:54.524216 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:58:54.524216 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:58:54.524216 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:58:54.524216 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:58:54.524216 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:58:54.524216 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:58:54.524216 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:58:54.524216 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:58:54.524216 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:58:54.524216 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:58:54.525619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:58:54.525619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:58:54.525619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:58:54.525619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:58:54.525619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:58:54.525619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:58:54.525619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:58:54.525619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:58:54.525619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:58:54.525619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:58:54.525619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:58:54.525619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:58:54.525619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 36/68] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 19:58:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:58:54.526855 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7c95d-bdaf-446c-802e-374c8a4e52d9 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:54.527422 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7c95d-bdaf-446c-802e-374c8a4e52d9 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:54.527708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7c95d-bdaf-446c-802e-374c8a4e52d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:54.551837 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-9ec7c95d-bdaf-446c-802e-374c8a4e52d9 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:58:54.552883 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-9ec7c95d-bdaf-446c-802e-374c8a4e52d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:58:54.562246 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce4172-505e-4a6a-bf38-7caf2dd01f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:54.562955 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce4172-505e-4a6a-bf38-7caf2dd01f31 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:54.563239 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce4172-505e-4a6a-bf38-7caf2dd01f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:54.563520 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce4172-505e-4a6a-bf38-7caf2dd01f31 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:54.564204 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce4172-505e-4a6a-bf38-7caf2dd01f31 None admin] RBAC: Adding query filter params (name=member) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:58:54.564824 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce4172-505e-4a6a-bf38-7caf2dd01f31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:54.565947 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce4172-505e-4a6a-bf38-7caf2dd01f31 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:54.575493 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 33/69] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:58:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:54.576706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f76ee8-7a4e-4133-b5ab-8ce5b00dee50 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:54.576998 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f76ee8-7a4e-4133-b5ab-8ce5b00dee50 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:54.577270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f76ee8-7a4e-4133-b5ab-8ce5b00dee50 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:54.578648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 34/70] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:58:54.579230 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f76ee8-7a4e-4133-b5ab-8ce5b00dee50 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:54.640985 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 37/71] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:58:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:58:54.641784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7c95d-bdaf-446c-802e-374c8a4e52d9 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:54.695973 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:54.707995 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f76ee8-7a4e-4133-b5ab-8ce5b00dee50 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:54.709588 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:54.749679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18700b-5876-4bdb-a857-9eb6447f1c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:54.750458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18700b-5876-4bdb-a857-9eb6447f1c0a None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:54.750742 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18700b-5876-4bdb-a857-9eb6447f1c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:54.751014 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18700b-5876-4bdb-a857-9eb6447f1c0a None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:54.751697 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b18700b-5876-4bdb-a857-9eb6447f1c0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:58:54.752274 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b18700b-5876-4bdb-a857-9eb6447f1c0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:54.753456 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b18700b-5876-4bdb-a857-9eb6447f1c0a None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:54.758097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7c95d-bdaf-446c-802e-374c8a4e52d9 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:54.763298 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 35/72] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:58:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:54.764403 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de496520-7c12-454e-8e30-2cfbd87453e7 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:54.764696 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de496520-7c12-454e-8e30-2cfbd87453e7 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:54.764977 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de496520-7c12-454e-8e30-2cfbd87453e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:54.784852 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-de496520-7c12-454e-8e30-2cfbd87453e7 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:58:54.786119 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-de496520-7c12-454e-8e30-2cfbd87453e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:58:54.804313 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2baa95-fe70-499b-b7aa-11e28caa5137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:54.805668 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2baa95-fe70-499b-b7aa-11e28caa5137 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:54.806158 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2baa95-fe70-499b-b7aa-11e28caa5137 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:54.806607 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2baa95-fe70-499b-b7aa-11e28caa5137 None admin] PATH_INFO: `/v3/users/b821a3d0f6354acaad7a12c771fb8141` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:54.823743 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2baa95-fe70-499b-b7aa-11e28caa5137 None admin] RBAC: Authorizing `identity:get_user(user_id=b821a3d0f6354acaad7a12c771fb8141)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:54.826318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2baa95-fe70-499b-b7aa-11e28caa5137 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:54.828268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 38/73] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:58:54 2022] GET /identity/v3/users/b821a3d0f6354acaad7a12c771fb8141 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:54.830558 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbc6f8-e2b4-428c-a673-8bf053c53266 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:54.831115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbc6f8-e2b4-428c-a673-8bf053c53266 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:54.832001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbc6f8-e2b4-428c-a673-8bf053c53266 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:54.834537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 39/74] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:58:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:58:54.840201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbc6f8-e2b4-428c-a673-8bf053c53266 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:54.861572 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 36/75] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:58:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:58:54.862110 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de496520-7c12-454e-8e30-2cfbd87453e7 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:54.975434 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:54.978941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbc6f8-e2b4-428c-a673-8bf053c53266 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:54.984564 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de496520-7c12-454e-8e30-2cfbd87453e7 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:54.988012 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:55.022248 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef77f1-b074-4652-9353-f1683f6e042f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:55.022866 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef77f1-b074-4652-9353-f1683f6e042f None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:55.023219 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef77f1-b074-4652-9353-f1683f6e042f None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:55.023419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef77f1-b074-4652-9353-f1683f6e042f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:55.025134 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3016a1c-16ba-47be-a7cf-e4075ed0cf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:55.026255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3016a1c-16ba-47be-a7cf-e4075ed0cf08 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:55.027295 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3016a1c-16ba-47be-a7cf-e4075ed0cf08 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:55.027928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3016a1c-16ba-47be-a7cf-e4075ed0cf08 None admin] PATH_INFO: `/v3/users/7ab6b80a7eaf40e691fcd95e0949b22e` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:55.030266 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3016a1c-16ba-47be-a7cf-e4075ed0cf08 None admin] RBAC: Authorizing `identity:get_user(user_id=7ab6b80a7eaf40e691fcd95e0949b22e)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:55.032811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3016a1c-16ba-47be-a7cf-e4075ed0cf08 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:55.035283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 40/76] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:58:54 2022] GET /identity/v3/users/7ab6b80a7eaf40e691fcd95e0949b22e => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:55.036434 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786f49b-764a-4098-b661-0fa239bf9c44 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:55.036718 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786f49b-764a-4098-b661-0fa239bf9c44 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:55.036995 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786f49b-764a-4098-b661-0fa239bf9c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:55.058357 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-f786f49b-764a-4098-b661-0fa239bf9c44 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:58:55.059608 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f786f49b-764a-4098-b661-0fa239bf9c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:58:55.110357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef77f1-b074-4652-9353-f1683f6e042f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:55.112565 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef77f1-b074-4652-9353-f1683f6e042f None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:55.140541 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 41/77] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:58:55.141485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786f49b-764a-4098-b661-0fa239bf9c44 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:55.219990 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:55.228317 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 37/78] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:58:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:58:55.234303 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef77f1-b074-4652-9353-f1683f6e042f None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:55.270788 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786f49b-764a-4098-b661-0fa239bf9c44 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:55.275184 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:55.314145 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62236c15-cc96-44d4-a42c-1963a3daa1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:55.314798 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62236c15-cc96-44d4-a42c-1963a3daa1bb None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:55.315081 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62236c15-cc96-44d4-a42c-1963a3daa1bb None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:55.315359 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62236c15-cc96-44d4-a42c-1963a3daa1bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:55.327701 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62236c15-cc96-44d4-a42c-1963a3daa1bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:58:55.332992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-62236c15-cc96-44d4-a42c-1963a3daa1bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:58:55.332992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:58:55.332992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:58:55.332992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:58:55.332992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:58:55.332992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:58:55.332992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:58:55.332992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:58:55.332992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:58:55.332992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:58:55.332992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:58:55.332992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:58:55.332992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:58:55.332992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:58:55.332992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:58:55.332992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:58:55.334138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:58:55.334138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:58:55.334138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:58:55.334138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:58:55.334138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:58:55.334138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:58:55.334138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:58:55.334138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:58:55.334138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:58:55.334138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:58:55.334138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:58:55.334138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:58:55.334138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:58:55.334138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:58:55.334138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:58:55.334138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:58:55.334893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:58:55.334893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:58:55.334893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:58:55.334893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:58:55.334893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:58:55.334893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:58:55.334893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:58:55.334893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:58:55.334893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:58:55.334893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:58:55.334893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:58:55.334893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:58:55.337113 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 42/79] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:58:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:58:55.338018 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62236c15-cc96-44d4-a42c-1963a3daa1bb None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:55.346091 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef77f1-b074-4652-9353-f1683f6e042f None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:55.385473 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5f244-3e13-49cb-a6e2-2a7cbb08ca2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:55.386439 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5f244-3e13-49cb-a6e2-2a7cbb08ca2e None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:55.386574 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5f244-3e13-49cb-a6e2-2a7cbb08ca2e None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:55.386928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5f244-3e13-49cb-a6e2-2a7cbb08ca2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:55.453430 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:55.454638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62236c15-cc96-44d4-a42c-1963a3daa1bb None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:55.461756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b5f244-3e13-49cb-a6e2-2a7cbb08ca2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:55.463254 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b5f244-3e13-49cb-a6e2-2a7cbb08ca2e None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:55.489989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b168f-6596-4ff3-9f5b-3c7ac0d34011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:55.490547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b168f-6596-4ff3-9f5b-3c7ac0d34011 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:55.490826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b168f-6596-4ff3-9f5b-3c7ac0d34011 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:55.491103 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b168f-6596-4ff3-9f5b-3c7ac0d34011 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:55.503260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9b168f-6596-4ff3-9f5b-3c7ac0d34011 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:58:55.508142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-ca9b168f-6596-4ff3-9f5b-3c7ac0d34011 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:58:55.508142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:58:55.508142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:58:55.508142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:58:55.508142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:58:55.508142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:58:55.508142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:58:55.508142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:58:55.508142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:58:55.508142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:58:55.508142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:58:55.508142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:58:55.508142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:58:55.508142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:58:55.508142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:58:55.508142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:58:55.509194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:58:55.509194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:58:55.509194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:58:55.509194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:58:55.509194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:58:55.509194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:58:55.509194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:58:55.509194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:58:55.509194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:58:55.509194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:58:55.509194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:58:55.509194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:58:55.509194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:58:55.509194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:58:55.509194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:58:55.509194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:58:55.510268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:58:55.510268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:58:55.510268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:58:55.510268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:58:55.510268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:58:55.510268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:58:55.510268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:58:55.510268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:58:55.510268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:58:55.510268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:58:55.510268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:58:55.510268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:58:55.510268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 43/80] 10.222.0.73 () {56 vars in 1114 bytes} [Mon Aug 29 19:58:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:58:55.510268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b168f-6596-4ff3-9f5b-3c7ac0d34011 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:55.538371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:55.571681 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 38/81] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:58:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:58:55.572765 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5f244-3e13-49cb-a6e2-2a7cbb08ca2e None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:55.619892 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b168f-6596-4ff3-9f5b-3c7ac0d34011 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:55.656926 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106e4b4-a303-4d92-b5f6-27895aafaabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:55.657567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106e4b4-a303-4d92-b5f6-27895aafaabb None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:55.657886 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106e4b4-a303-4d92-b5f6-27895aafaabb None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:55.658162 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106e4b4-a303-4d92-b5f6-27895aafaabb None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:55.659320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106e4b4-a303-4d92-b5f6-27895aafaabb None admin] RBAC: Authorizing `identity:get_project(project_id=4367ec2ce611409a85975a2a505f1ff8)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:55.660788 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106e4b4-a303-4d92-b5f6-27895aafaabb None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:55.662519 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 44/82] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:58:55 2022] GET /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:55.663288 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106e4b4-a303-4d92-b5f6-27895aafaabb None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:55.679078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:55.687908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5f244-3e13-49cb-a6e2-2a7cbb08ca2e None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:55.724617 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6c316-a0df-4b78-84f7-5d8dd962f862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:55.725230 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6c316-a0df-4b78-84f7-5d8dd962f862 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:55.725511 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6c316-a0df-4b78-84f7-5d8dd962f862 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:55.725814 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6c316-a0df-4b78-84f7-5d8dd962f862 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:55.726490 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd6c316-a0df-4b78-84f7-5d8dd962f862 None admin] RBAC: Adding query filter params (name=member) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:58:55.726792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd6c316-a0df-4b78-84f7-5d8dd962f862 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:55.728157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd6c316-a0df-4b78-84f7-5d8dd962f862 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:55.737097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 39/83] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:58:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:55.740149 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6c316-a0df-4b78-84f7-5d8dd962f862 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:55.762680 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:55.771889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106e4b4-a303-4d92-b5f6-27895aafaabb None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:55.810169 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417788c-a443-4608-b175-bb40320f4030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:55.810823 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417788c-a443-4608-b175-bb40320f4030 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:55.811110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417788c-a443-4608-b175-bb40320f4030 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:55.811384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417788c-a443-4608-b175-bb40320f4030 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:55.812707 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9417788c-a443-4608-b175-bb40320f4030 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:55.814260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9417788c-a443-4608-b175-bb40320f4030 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:55.815954 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 45/84] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:58:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:55.816755 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417788c-a443-4608-b175-bb40320f4030 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:55.850392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6c316-a0df-4b78-84f7-5d8dd962f862 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:55.888404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3df74-726a-4526-80ef-18e38da51a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:55.889017 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3df74-726a-4526-80ef-18e38da51a46 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:55.889308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3df74-726a-4526-80ef-18e38da51a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:55.889577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3df74-726a-4526-80ef-18e38da51a46 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:55.890272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3df74-726a-4526-80ef-18e38da51a46 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:58:55.890573 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3df74-726a-4526-80ef-18e38da51a46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:55.891946 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3df74-726a-4526-80ef-18e38da51a46 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:55.900387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 40/85] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 19:58:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:55.901200 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3df74-726a-4526-80ef-18e38da51a46 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:55.923285 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:55.936512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417788c-a443-4608-b175-bb40320f4030 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:55.973945 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e80acf-e764-4389-803b-3988f1ef3a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:55.974583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e80acf-e764-4389-803b-3988f1ef3a6f None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:55.974863 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e80acf-e764-4389-803b-3988f1ef3a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:55.975234 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e80acf-e764-4389-803b-3988f1ef3a6f None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:55.976291 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e80acf-e764-4389-803b-3988f1ef3a6f None admin] RBAC: Authorizing `identity:get_project(project_id=4367ec2ce611409a85975a2a505f1ff8)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:55.977759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e80acf-e764-4389-803b-3988f1ef3a6f None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:55.979404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 46/86] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:58:55 2022] GET /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:55.985200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e80acf-e764-4389-803b-3988f1ef3a6f None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:56.013942 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3df74-726a-4526-80ef-18e38da51a46 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:56.014133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:56.047877 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe64e43-bb37-48ce-ae5b-91ddf336bb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:56.048487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe64e43-bb37-48ce-ae5b-91ddf336bb27 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:56.048773 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe64e43-bb37-48ce-ae5b-91ddf336bb27 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:56.049046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe64e43-bb37-48ce-ae5b-91ddf336bb27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:56.049875 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe64e43-bb37-48ce-ae5b-91ddf336bb27 None admin] RBAC: Adding query filter params (role.id=9265e1fd617b4c0b8fbf267885394060, scope.project.id=4367ec2ce611409a85975a2a505f1ff8, user.id=b821a3d0f6354acaad7a12c771fb8141) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:58:56.050133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe64e43-bb37-48ce-ae5b-91ddf336bb27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:56.051598 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe64e43-bb37-48ce-ae5b-91ddf336bb27 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:56.059212 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 41/87] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 19:58:55 2022] GET /identity/v3/role_assignments?user.id=b821a3d0f6354acaad7a12c771fb8141&scope.project.id=4367ec2ce611409a85975a2a505f1ff8&role.id=9265e1fd617b4c0b8fbf267885394060 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:56.060636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe64e43-bb37-48ce-ae5b-91ddf336bb27 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:56.101257 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e80acf-e764-4389-803b-3988f1ef3a6f None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:56.145421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbb01f-b8d3-436a-bf45-efcdfaa7f2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:56.146732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbb01f-b8d3-436a-bf45-efcdfaa7f2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:56.147339 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbb01f-b8d3-436a-bf45-efcdfaa7f2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:56.147840 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbb01f-b8d3-436a-bf45-efcdfaa7f2c1 None admin] PATH_INFO: `/v3/users/b821a3d0f6354acaad7a12c771fb8141` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:56.150083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbbb01f-b8d3-436a-bf45-efcdfaa7f2c1 None admin] RBAC: Authorizing `identity:get_user(user_id=b821a3d0f6354acaad7a12c771fb8141)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:56.153696 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbbb01f-b8d3-436a-bf45-efcdfaa7f2c1 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:56.156949 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 47/88] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:58:55 2022] GET /identity/v3/users/b821a3d0f6354acaad7a12c771fb8141 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:56.158566 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbb01f-b8d3-436a-bf45-efcdfaa7f2c1 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:56.179353 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:56.200117 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe64e43-bb37-48ce-ae5b-91ddf336bb27 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:56.243043 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f36b5-812b-472b-9bc2-8f3a58784da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:56.243935 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f36b5-812b-472b-9bc2-8f3a58784da0 None admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:56.244221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f36b5-812b-472b-9bc2-8f3a58784da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:56.244586 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f36b5-812b-472b-9bc2-8f3a58784da0 None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:56.245685 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08f36b5-812b-472b-9bc2-8f3a58784da0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:56.247186 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08f36b5-812b-472b-9bc2-8f3a58784da0 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:56.271006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 42/89] 10.222.0.73 () {60 vars in 1132 bytes} [Mon Aug 29 19:58:56 2022] POST /identity/v3/users => generated 289 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:58:56.271771 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f36b5-812b-472b-9bc2-8f3a58784da0 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:56.286996 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:56.299012 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbb01f-b8d3-436a-bf45-efcdfaa7f2c1 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:56.341687 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525916c3-a177-466b-bbb9-1f2f33bfa349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:56.342530 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525916c3-a177-466b-bbb9-1f2f33bfa349 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:56.343128 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525916c3-a177-466b-bbb9-1f2f33bfa349 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:56.343482 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525916c3-a177-466b-bbb9-1f2f33bfa349 None admin] PATH_INFO: `/v3/users/b821a3d0f6354acaad7a12c771fb8141` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:56.344951 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525916c3-a177-466b-bbb9-1f2f33bfa349 None admin] RBAC: Authorizing `identity:get_user(user_id=b821a3d0f6354acaad7a12c771fb8141)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:56.346639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525916c3-a177-466b-bbb9-1f2f33bfa349 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:56.348816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 48/90] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:58:56 2022] GET /identity/v3/users/b821a3d0f6354acaad7a12c771fb8141 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:56.354946 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525916c3-a177-466b-bbb9-1f2f33bfa349 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:56.430653 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f36b5-812b-472b-9bc2-8f3a58784da0 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:56.473524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff92b0d-e304-43e5-89de-922864ffaf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:56.474604 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff92b0d-e304-43e5-89de-922864ffaf39 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:56.475162 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff92b0d-e304-43e5-89de-922864ffaf39 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:56.475632 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff92b0d-e304-43e5-89de-922864ffaf39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:56.476537 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff92b0d-e304-43e5-89de-922864ffaf39 None admin] RBAC: Adding query filter params (role.id=296ee2c194824485af71618619f82e09, scope.project.id=4367ec2ce611409a85975a2a505f1ff8, user.id=7ab6b80a7eaf40e691fcd95e0949b22e) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:58:56.476853 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff92b0d-e304-43e5-89de-922864ffaf39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:56.478421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff92b0d-e304-43e5-89de-922864ffaf39 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:56.479347 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525916c3-a177-466b-bbb9-1f2f33bfa349 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:56.485255 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 43/91] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 19:58:56 2022] GET /identity/v3/role_assignments?user.id=7ab6b80a7eaf40e691fcd95e0949b22e&scope.project.id=4367ec2ce611409a85975a2a505f1ff8&role.id=296ee2c194824485af71618619f82e09 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:56.486138 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff92b0d-e304-43e5-89de-922864ffaf39 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:56.490817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:56.529467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbbd9b-d019-4df9-9cfa-afaf9ee78126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:56.530071 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbbd9b-d019-4df9-9cfa-afaf9ee78126 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:56.530425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbbd9b-d019-4df9-9cfa-afaf9ee78126 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:56.530704 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbbd9b-d019-4df9-9cfa-afaf9ee78126 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:56.580448 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:56.626314 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbbd9b-d019-4df9-9cfa-afaf9ee78126 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:56.627920 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbbd9b-d019-4df9-9cfa-afaf9ee78126 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:56.634746 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff92b0d-e304-43e5-89de-922864ffaf39 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:56.686787 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c136f-9cf2-460d-84b7-82665c08d4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:56.687465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c136f-9cf2-460d-84b7-82665c08d4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:56.687751 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c136f-9cf2-460d-84b7-82665c08d4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:56.688027 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c136f-9cf2-460d-84b7-82665c08d4d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:56.773432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 49/92] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:58:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:58:56.780115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbbd9b-d019-4df9-9cfa-afaf9ee78126 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:56.784958 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c136f-9cf2-460d-84b7-82665c08d4d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:56.786479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c136f-9cf2-460d-84b7-82665c08d4d7 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:56.790034 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:56.865179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:56.912623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 44/93] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:58:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:58:56.918663 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c136f-9cf2-460d-84b7-82665c08d4d7 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:56.935953 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbbd9b-d019-4df9-9cfa-afaf9ee78126 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:57.004576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27363b-f99e-4bd8-ac05-0bec9877bd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:57.005226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27363b-f99e-4bd8-ac05-0bec9877bd15 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:57.005508 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27363b-f99e-4bd8-ac05-0bec9877bd15 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:57.005808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27363b-f99e-4bd8-ac05-0bec9877bd15 None admin] PATH_INFO: `/v3/projects/4836001ac84c48758f3fcbff0cd12afc` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:57.007851 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27363b-f99e-4bd8-ac05-0bec9877bd15 None admin] RBAC: Authorizing `identity:get_project(project_id=4836001ac84c48758f3fcbff0cd12afc)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:57.009345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27363b-f99e-4bd8-ac05-0bec9877bd15 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:57.012018 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 50/94] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:58:56 2022] GET /identity/v3/projects/4836001ac84c48758f3fcbff0cd12afc => generated 312 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:58:57.018698 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27363b-f99e-4bd8-ac05-0bec9877bd15 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:57.101168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c136f-9cf2-460d-84b7-82665c08d4d7 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:57.101852 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:57.143778 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274057f-6738-430e-841d-3e66e4f20fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:57.144432 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274057f-6738-430e-841d-3e66e4f20fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:57.144720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274057f-6738-430e-841d-3e66e4f20fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:57.144999 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274057f-6738-430e-841d-3e66e4f20fc3 None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:57.153336 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274057f-6738-430e-841d-3e66e4f20fc3 None admin] RBAC: Authorizing `identity:get_project(project_id=4367ec2ce611409a85975a2a505f1ff8)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:57.154867 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274057f-6738-430e-841d-3e66e4f20fc3 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:57.158202 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 45/95] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:58:56 2022] GET /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:58:57.163872 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274057f-6738-430e-841d-3e66e4f20fc3 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:57.239124 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27363b-f99e-4bd8-ac05-0bec9877bd15 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:57.297797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:57.317042 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab161a2-51cd-4ade-b7ee-66155caacd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:57.321339 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab161a2-51cd-4ade-b7ee-66155caacd31 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:57.321882 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab161a2-51cd-4ade-b7ee-66155caacd31 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:57.322354 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab161a2-51cd-4ade-b7ee-66155caacd31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:57.323997 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab161a2-51cd-4ade-b7ee-66155caacd31 None admin] RBAC: Adding query filter params (role.id=9265e1fd617b4c0b8fbf267885394060, scope.project.id=4836001ac84c48758f3fcbff0cd12afc, user.id=b821a3d0f6354acaad7a12c771fb8141) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:58:57.324613 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab161a2-51cd-4ade-b7ee-66155caacd31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:57.327624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab161a2-51cd-4ade-b7ee-66155caacd31 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:57.342456 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 51/96] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 19:58:57 2022] GET /identity/v3/role_assignments?user.id=b821a3d0f6354acaad7a12c771fb8141&scope.project.id=4836001ac84c48758f3fcbff0cd12afc&role.id=9265e1fd617b4c0b8fbf267885394060 => generated 259 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:57.386431 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274057f-6738-430e-841d-3e66e4f20fc3 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:57.443180 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff492a-5859-4184-a58d-af89a977ddae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:57.443816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff492a-5859-4184-a58d-af89a977ddae None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:57.444105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff492a-5859-4184-a58d-af89a977ddae None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:57.444378 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff492a-5859-4184-a58d-af89a977ddae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:57.445131 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff492a-5859-4184-a58d-af89a977ddae None admin] RBAC: Adding query filter params (role.id=b9184599d95b4950beca34304cdd2eb8, scope.project.id=4367ec2ce611409a85975a2a505f1ff8, user.id=b821a3d0f6354acaad7a12c771fb8141) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:58:57.445443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff492a-5859-4184-a58d-af89a977ddae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:57.446992 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff492a-5859-4184-a58d-af89a977ddae None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:57.458398 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 46/97] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 19:58:57 2022] GET /identity/v3/role_assignments?user.id=b821a3d0f6354acaad7a12c771fb8141&scope.project.id=4367ec2ce611409a85975a2a505f1ff8&role.id=b9184599d95b4950beca34304cdd2eb8 => generated 259 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:58.503029 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b348a52-3bb9-4fa4-8dad-01a0d2c7a6d5 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:58.503567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b348a52-3bb9-4fa4-8dad-01a0d2c7a6d5 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:58.503567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b348a52-3bb9-4fa4-8dad-01a0d2c7a6d5 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:58.504892 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 52/98] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:58:58.520021 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a983b8f-a176-4b44-aca0-5053cdf19ac9 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:58.520540 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a983b8f-a176-4b44-aca0-5053cdf19ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:58.520643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a983b8f-a176-4b44-aca0-5053cdf19ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:58.551642 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-6a983b8f-a176-4b44-aca0-5053cdf19ac9 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:58:58.552865 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-6a983b8f-a176-4b44-aca0-5053cdf19ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:58:58.592650 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:58.646427 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 47/99] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:58:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:58:59.046548 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba9a77-3192-4fab-bb42-ab4359e0b066 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:59.047197 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba9a77-3192-4fab-bb42-ab4359e0b066 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:59.047485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba9a77-3192-4fab-bb42-ab4359e0b066 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:59.049118 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 53/100] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:58:59.054350 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef841ea-3dbf-4a61-b317-893fe2894efd None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:59.054642 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef841ea-3dbf-4a61-b317-893fe2894efd None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:59.054911 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef841ea-3dbf-4a61-b317-893fe2894efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:59.078767 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-7ef841ea-3dbf-4a61-b317-893fe2894efd None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:58:59.080092 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-7ef841ea-3dbf-4a61-b317-893fe2894efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:58:59.118548 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba9a77-3192-4fab-bb42-ab4359e0b066 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:59.215146 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 48/101] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:58:59.284721 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba9a77-3192-4fab-bb42-ab4359e0b066 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:59.330937 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a0353-7576-469e-9f1f-628c13343860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:59.331641 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a0353-7576-469e-9f1f-628c13343860 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:59.331929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a0353-7576-469e-9f1f-628c13343860 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:59.332205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a0353-7576-469e-9f1f-628c13343860 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:59.347456 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a0353-7576-469e-9f1f-628c13343860 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:58:59.353457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-522a0353-7576-469e-9f1f-628c13343860 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:58:59.353457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:58:59.353457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:58:59.353457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:58:59.353457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:58:59.353457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:58:59.353457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:58:59.353457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:58:59.353457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:58:59.353457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:58:59.353457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:58:59.353457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:58:59.353457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:58:59.353457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:58:59.353457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:58:59.353457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:58:59.354553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:58:59.354553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:58:59.354553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:58:59.354553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:58:59.354553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:58:59.354553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:58:59.354553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:58:59.354553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:58:59.354553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:58:59.354553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:58:59.354553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:58:59.354553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:58:59.354553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:58:59.354553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:58:59.354553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:58:59.354553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:58:59.355583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:58:59.355583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:58:59.355583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:58:59.355583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:58:59.355583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:58:59.355583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:58:59.355583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:58:59.355583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:58:59.355583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:58:59.355583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:58:59.355583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:58:59.355583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:58:59.355583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 54/102] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:58:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 19:58:59.363642 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef841ea-3dbf-4a61-b317-893fe2894efd None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:59.510526 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:59.518047 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef841ea-3dbf-4a61-b317-893fe2894efd None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:59.572980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7bb07-a3ff-4d7c-b98a-eb3b8af05f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:59.573896 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7bb07-a3ff-4d7c-b98a-eb3b8af05f0f None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:59.574264 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7bb07-a3ff-4d7c-b98a-eb3b8af05f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:59.574534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7bb07-a3ff-4d7c-b98a-eb3b8af05f0f None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:59.575227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7bb07-a3ff-4d7c-b98a-eb3b8af05f0f None admin] RBAC: Adding query filter params (name=member) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:58:59.575574 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7bb07-a3ff-4d7c-b98a-eb3b8af05f0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:59.577037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7bb07-a3ff-4d7c-b98a-eb3b8af05f0f None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:59.586187 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 49/103] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:58:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:59.592610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a0353-7576-469e-9f1f-628c13343860 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:59.659452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:59.685777 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64135804-e018-4e6b-8b0d-0f25d42e73c4 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:59.686372 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64135804-e018-4e6b-8b0d-0f25d42e73c4 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:59.686819 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64135804-e018-4e6b-8b0d-0f25d42e73c4 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:59.688792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 50/104] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:58:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:58:59.694639 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9908ad9-c66c-4ae6-9e46-0699a48ecc0f None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:59.695126 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9908ad9-c66c-4ae6-9e46-0699a48ecc0f None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:59.695970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9908ad9-c66c-4ae6-9e46-0699a48ecc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:59.728002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-c9908ad9-c66c-4ae6-9e46-0699a48ecc0f None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:58:59.729222 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c9908ad9-c66c-4ae6-9e46-0699a48ecc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:58:59.734751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a0353-7576-469e-9f1f-628c13343860 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:58:59.822804 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 51/105] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:58:59.824257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9908ad9-c66c-4ae6-9e46-0699a48ecc0f None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:59.826448 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6a2d7-6e3a-4cb9-aef9-e4bf44dafda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:58:59.827128 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6a2d7-6e3a-4cb9-aef9-e4bf44dafda5 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:58:59.827420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6a2d7-6e3a-4cb9-aef9-e4bf44dafda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:58:59.827704 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6a2d7-6e3a-4cb9-aef9-e4bf44dafda5 None admin] PATH_INFO: `/v3/users/b821a3d0f6354acaad7a12c771fb8141` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:58:59.829746 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6a2d7-6e3a-4cb9-aef9-e4bf44dafda5 None admin] RBAC: Authorizing `identity:get_user(user_id=b821a3d0f6354acaad7a12c771fb8141)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:58:59.831237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6a2d7-6e3a-4cb9-aef9-e4bf44dafda5 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:58:59.832906 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 55/106] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:58:59 2022] GET /identity/v3/users/b821a3d0f6354acaad7a12c771fb8141 => generated 275 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:58:59.845994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6a2d7-6e3a-4cb9-aef9-e4bf44dafda5 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:58:59.982259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:58:59.991951 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9908ad9-c66c-4ae6-9e46-0699a48ecc0f None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:00.065593 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662f451-9bc7-49a0-9a17-a71238ea5467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:00.067650 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662f451-9bc7-49a0-9a17-a71238ea5467 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:00.067936 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662f451-9bc7-49a0-9a17-a71238ea5467 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:00.068208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662f451-9bc7-49a0-9a17-a71238ea5467 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:00.070736 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6a2d7-6e3a-4cb9-aef9-e4bf44dafda5 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:00.093760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e662f451-9bc7-49a0-9a17-a71238ea5467 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:00.094347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-e662f451-9bc7-49a0-9a17-a71238ea5467 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:59:00.094347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:00.094347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:00.094347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:00.094347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:00.094347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:00.094347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:00.094347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:00.094347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:00.094347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:00.094347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:00.094347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:00.094347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:00.094347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:00.094347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:00.094347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:00.095150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:00.095150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:00.095150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:00.095150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:00.095150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:00.095150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:00.095150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:00.095150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:00.095150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:00.095150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:00.095150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:00.095150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:00.095150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:00.095150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:00.095150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:00.095150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:00.095913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:00.095913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:00.095913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:00.095913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:00.095913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:00.095913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:00.095913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:00.095913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:00.095913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:00.095913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:00.095913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:59:00.095913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:00.099887 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 52/107] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 19:58:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:00.099887 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a78222-9bf4-48e1-97aa-654ce5922a4b None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:00.099887 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a78222-9bf4-48e1-97aa-654ce5922a4b None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:00.100129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a78222-9bf4-48e1-97aa-654ce5922a4b None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:00.114760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 53/108] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:00 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:00.115422 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a78222-9bf4-48e1-97aa-654ce5922a4b None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:00.126912 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:00.151256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941f03f-9f89-49b9-a3ca-7f1b6158c00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:00.152683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941f03f-9f89-49b9-a3ca-7f1b6158c00e None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:00.153012 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941f03f-9f89-49b9-a3ca-7f1b6158c00e None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:00.153293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941f03f-9f89-49b9-a3ca-7f1b6158c00e None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:00.155745 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0941f03f-9f89-49b9-a3ca-7f1b6158c00e None admin] RBAC: Authorizing `identity:get_project(project_id=4367ec2ce611409a85975a2a505f1ff8)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:00.158645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0941f03f-9f89-49b9-a3ca-7f1b6158c00e None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:00.162448 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 56/109] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:58:59 2022] GET /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8 => generated 298 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:00.164731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c6ba8-d606-4ef6-8e81-4f3e04237cb1 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:00.165014 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c6ba8-d606-4ef6-8e81-4f3e04237cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:00.166879 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c6ba8-d606-4ef6-8e81-4f3e04237cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:00.207845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-185c6ba8-d606-4ef6-8e81-4f3e04237cb1 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:00.209124 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-185c6ba8-d606-4ef6-8e81-4f3e04237cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:00.291527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a78222-9bf4-48e1-97aa-654ce5922a4b None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:00.303509 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 57/110] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:00.306814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c6ba8-d606-4ef6-8e81-4f3e04237cb1 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:00.362806 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2281a589-9c94-4bb0-b8b9-ee5b9461a4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:00.363779 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281a589-9c94-4bb0-b8b9-ee5b9461a4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:00.364258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281a589-9c94-4bb0-b8b9-ee5b9461a4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:00.364726 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281a589-9c94-4bb0-b8b9-ee5b9461a4d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:00.365607 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2281a589-9c94-4bb0-b8b9-ee5b9461a4d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:00.366139 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2281a589-9c94-4bb0-b8b9-ee5b9461a4d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:00.369347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2281a589-9c94-4bb0-b8b9-ee5b9461a4d7 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:00.380536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 54/111] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:00.381781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2281a589-9c94-4bb0-b8b9-ee5b9461a4d7 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:00.396289 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:00.450019 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c6ba8-d606-4ef6-8e81-4f3e04237cb1 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:00.475720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:00.489501 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8b7ed-13ff-41b9-b578-c73fb1e11bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:00.490558 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8b7ed-13ff-41b9-b578-c73fb1e11bc2 None admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:00.491074 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8b7ed-13ff-41b9-b578-c73fb1e11bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:00.491510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8b7ed-13ff-41b9-b578-c73fb1e11bc2 None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8/users/b821a3d0f6354acaad7a12c771fb8141/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:00.494150 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca8b7ed-13ff-41b9-b578-c73fb1e11bc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4367ec2ce611409a85975a2a505f1ff8, user_id=b821a3d0f6354acaad7a12c771fb8141, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:00.495821 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca8b7ed-13ff-41b9-b578-c73fb1e11bc2 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:00.508252 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2281a589-9c94-4bb0-b8b9-ee5b9461a4d7 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:00.512851 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 58/112] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 19:59:00 2022] PUT /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8/users/b821a3d0f6354acaad7a12c771fb8141/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:59:00.514273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c23d0c-8326-495f-9903-1ea285189bbf None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:00.514700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c23d0c-8326-495f-9903-1ea285189bbf None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:00.515079 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c23d0c-8326-495f-9903-1ea285189bbf None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:00.516755 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 59/113] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:00.517491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c23d0c-8326-495f-9903-1ea285189bbf None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:00.552625 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba272885-cc38-4e50-a017-ed5ebec85011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:00.553781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba272885-cc38-4e50-a017-ed5ebec85011 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:00.554306 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba272885-cc38-4e50-a017-ed5ebec85011 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:00.554763 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba272885-cc38-4e50-a017-ed5ebec85011 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:00.556387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba272885-cc38-4e50-a017-ed5ebec85011 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:00.559082 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba272885-cc38-4e50-a017-ed5ebec85011 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:00.566187 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 55/114] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:00.568152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0dc84-f343-40c3-b9f9-b4ff6c822cc6 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:00.568671 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0dc84-f343-40c3-b9f9-b4ff6c822cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:00.569157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0dc84-f343-40c3-b9f9-b4ff6c822cc6 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:00.571757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 56/115] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:00.573746 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfe8d4-5e99-4613-bf79-d3e85b4af3b9 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:00.574770 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfe8d4-5e99-4613-bf79-d3e85b4af3b9 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:00.575276 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfe8d4-5e99-4613-bf79-d3e85b4af3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:00.596897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-57cfe8d4-5e99-4613-bf79-d3e85b4af3b9 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:00.598198 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-57cfe8d4-5e99-4613-bf79-d3e85b4af3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:00.663421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c23d0c-8326-495f-9903-1ea285189bbf None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:00.689898 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 57/116] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:00.690859 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cfe8d4-5e99-4613-bf79-d3e85b4af3b9 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:00.702164 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:00.709192 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9df81d-e8c0-4f62-96bc-b4c9d5bc88e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:00.709893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9df81d-e8c0-4f62-96bc-b4c9d5bc88e0 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:00.710177 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9df81d-e8c0-4f62-96bc-b4c9d5bc88e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:00.710511 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9df81d-e8c0-4f62-96bc-b4c9d5bc88e0 None admin] PATH_INFO: `/v3/users/7ab6b80a7eaf40e691fcd95e0949b22e` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:00.711874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9df81d-e8c0-4f62-96bc-b4c9d5bc88e0 None admin] RBAC: Authorizing `identity:get_user(user_id=7ab6b80a7eaf40e691fcd95e0949b22e)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:00.713593 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9df81d-e8c0-4f62-96bc-b4c9d5bc88e0 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:00.715864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 60/117] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:00 2022] GET /identity/v3/users/7ab6b80a7eaf40e691fcd95e0949b22e => generated 247 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:00.717244 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847534b-8a85-44e6-a276-e90a5af27a5c None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:00.717529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847534b-8a85-44e6-a276-e90a5af27a5c None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:00.717843 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847534b-8a85-44e6-a276-e90a5af27a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:00.740588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-2847534b-8a85-44e6-a276-e90a5af27a5c None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:00.741888 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-2847534b-8a85-44e6-a276-e90a5af27a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:00.798820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:00.823515 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 61/118] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:00.824198 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cfe8d4-5e99-4613-bf79-d3e85b4af3b9 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:00.830681 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847534b-8a85-44e6-a276-e90a5af27a5c None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:00.865770 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a178a-37b4-472e-828f-1575145bf4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:00.866687 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a178a-37b4-472e-828f-1575145bf4f3 None admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:00.867249 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a178a-37b4-472e-828f-1575145bf4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:00.867694 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a178a-37b4-472e-828f-1575145bf4f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:00.868525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a178a-37b4-472e-828f-1575145bf4f3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:00.871601 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a178a-37b4-472e-828f-1575145bf4f3 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:00.913875 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 58/119] 10.222.0.73 () {60 vars in 1141 bytes} [Mon Aug 29 19:59:00 2022] POST /identity/v3/projects => generated 302 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:59:00.915263 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a178a-37b4-472e-828f-1575145bf4f3 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:00.980094 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847534b-8a85-44e6-a276-e90a5af27a5c None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:00.983481 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:01.025317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218b3f6-0748-4f03-a599-85d5f8162936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:01.029759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218b3f6-0748-4f03-a599-85d5f8162936 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:01.030075 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218b3f6-0748-4f03-a599-85d5f8162936 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:01.030417 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218b3f6-0748-4f03-a599-85d5f8162936 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:01.032505 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a178a-37b4-472e-828f-1575145bf4f3 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:01.043972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8218b3f6-0748-4f03-a599-85d5f8162936 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:01.050007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-8218b3f6-0748-4f03-a599-85d5f8162936 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 19:59:01.050007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:01.050007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:01.050007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:01.050007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:01.050007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:01.050007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:01.050007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:01.050007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:01.050007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:01.050007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:01.050007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:01.050007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:01.050007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:01.050007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:01.050007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:01.050873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:01.050873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:01.050873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:01.050873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:01.050873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:01.050873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:01.050873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:01.050873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:01.050873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:01.050873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:01.050873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:01.050873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:01.050873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:01.050873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:01.050873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:01.050873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:01.051612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:01.051612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:01.051612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:01.051612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:01.051612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:01.051612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:01.051612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:01.051612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:01.051612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:01.051612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:01.051612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 19:59:01.051612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:01.051612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 62/120] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:01.052243 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218b3f6-0748-4f03-a599-85d5f8162936 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:01.093414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7caaaf-2f53-4d1a-9f4e-304b8fb87795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:01.095458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7caaaf-2f53-4d1a-9f4e-304b8fb87795 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:01.095919 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7caaaf-2f53-4d1a-9f4e-304b8fb87795 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:01.096321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7caaaf-2f53-4d1a-9f4e-304b8fb87795 None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:01.101045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7caaaf-2f53-4d1a-9f4e-304b8fb87795 None admin] RBAC: Authorizing `identity:get_project(project_id=4367ec2ce611409a85975a2a505f1ff8)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:01.107124 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7caaaf-2f53-4d1a-9f4e-304b8fb87795 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:01.109381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 59/121] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:00 2022] GET /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:01.113457 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7caaaf-2f53-4d1a-9f4e-304b8fb87795 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:01.141040 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:01.226682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218b3f6-0748-4f03-a599-85d5f8162936 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:01.263215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7caaaf-2f53-4d1a-9f4e-304b8fb87795 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:01.280144 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6de43-89e6-48f8-b8ad-2960c9ee5eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:01.280797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6de43-89e6-48f8-b8ad-2960c9ee5eff None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:01.281334 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6de43-89e6-48f8-b8ad-2960c9ee5eff None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:01.281610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6de43-89e6-48f8-b8ad-2960c9ee5eff None admin] PATH_INFO: `/v3/roles/member` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:01.291325 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:01.299826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed6de43-89e6-48f8-b8ad-2960c9ee5eff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:01.302961 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8c7c1-435b-4198-8903-a330fff1b51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:01.303578 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8c7c1-435b-4198-8903-a330fff1b51b None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:01.303866 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8c7c1-435b-4198-8903-a330fff1b51b None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:01.304147 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8c7c1-435b-4198-8903-a330fff1b51b None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:01.304751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-9ed6de43-89e6-48f8-b8ad-2960c9ee5eff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:01.304751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:01.304751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:01.304751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:01.304751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:01.304751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:01.304751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:01.304751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:01.304751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:01.304751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:01.304751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:01.304751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:01.304751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:01.304751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:01.304751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:01.304751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:01.305586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:01.305586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:01.305586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:01.305586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:01.305586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:01.305586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:01.305586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:01.305586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:01.305586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:01.305586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:01.305586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:01.305586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:01.305586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:01.305586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:01.305586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:01.305586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:01.306391 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:01.306391 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:01.306391 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:01.306391 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:01.306391 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:01.306391 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:01.306391 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:01.306391 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:01.306391 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:01.306391 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:01.306391 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:01.306391 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:01.306972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c8c7c1-435b-4198-8903-a330fff1b51b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:01.306972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c8c7c1-435b-4198-8903-a330fff1b51b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:01.307086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 63/122] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:01.307086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6de43-89e6-48f8-b8ad-2960c9ee5eff None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:01.307205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c8c7c1-435b-4198-8903-a330fff1b51b None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:01.324869 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 60/123] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:01.325768 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8c7c1-435b-4198-8903-a330fff1b51b None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:01.444903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:01.449371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6de43-89e6-48f8-b8ad-2960c9ee5eff None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:01.452492 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8c7c1-435b-4198-8903-a330fff1b51b None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:01.489180 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f760f-bf55-4f64-8762-ab84e66d7b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:01.489798 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f760f-bf55-4f64-8762-ab84e66d7b46 None admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:01.490321 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f760f-bf55-4f64-8762-ab84e66d7b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:01.490464 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f760f-bf55-4f64-8762-ab84e66d7b46 None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8/users/7ab6b80a7eaf40e691fcd95e0949b22e/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:01.492572 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f760f-bf55-4f64-8762-ab84e66d7b46 None admin] RBAC: Authorizing `identity:create_grant(project_id=4367ec2ce611409a85975a2a505f1ff8, user_id=7ab6b80a7eaf40e691fcd95e0949b22e, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:01.494519 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f760f-bf55-4f64-8762-ab84e66d7b46 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:01.500477 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a916df-fe3b-4537-bee5-37e72f54b88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:01.501470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a916df-fe3b-4537-bee5-37e72f54b88a None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:01.501866 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a916df-fe3b-4537-bee5-37e72f54b88a None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:01.502186 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a916df-fe3b-4537-bee5-37e72f54b88a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:01.510888 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 64/124] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 19:59:01 2022] PUT /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8/users/7ab6b80a7eaf40e691fcd95e0949b22e/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:59:01.512067 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f760f-bf55-4f64-8762-ab84e66d7b46 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:01.513694 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a916df-fe3b-4537-bee5-37e72f54b88a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:01.519064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-20a916df-fe3b-4537-bee5-37e72f54b88a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:01.519064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:01.519064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:01.519064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:01.519064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:01.519064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:01.519064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:01.519064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:01.519064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:01.519064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:01.519064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:01.519064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:01.519064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:01.519064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:01.519064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:01.519064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:01.520054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:01.520054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:01.520054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:01.520054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:01.520054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:01.520054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:01.520054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:01.520054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:01.520054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:01.520054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:01.520054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:01.520054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:01.520054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:01.520054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:01.520054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:01.520054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:01.520882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:01.520882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:01.520882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:01.520882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:01.520882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:01.520882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:01.520882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:01.520882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:01.520882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:01.520882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:01.520882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:01.520882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:01.520882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 61/125] 10.222.0.73 () {56 vars in 1114 bytes} [Mon Aug 29 19:59:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:01.521512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a916df-fe3b-4537-bee5-37e72f54b88a None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:01.585537 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:01.644004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f760f-bf55-4f64-8762-ab84e66d7b46 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:01.647543 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a916df-fe3b-4537-bee5-37e72f54b88a None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:01.685622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87910595-0a0f-4538-8e08-2a2a00740977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:01.686979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87910595-0a0f-4538-8e08-2a2a00740977 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:01.687307 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3920dc-409e-467f-a4b2-5f45dc864f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:01.687736 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87910595-0a0f-4538-8e08-2a2a00740977 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:01.687961 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3920dc-409e-467f-a4b2-5f45dc864f49 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:01.688246 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3920dc-409e-467f-a4b2-5f45dc864f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:01.688521 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3920dc-409e-467f-a4b2-5f45dc864f49 None admin] PATH_INFO: `/v3/users/abf0cae0793b4db39afb4a7f1e5b6035` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:01.688821 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87910595-0a0f-4538-8e08-2a2a00740977 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:01.689737 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87910595-0a0f-4538-8e08-2a2a00740977 None admin] RBAC: Adding query filter params (name=member) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:01.690254 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87910595-0a0f-4538-8e08-2a2a00740977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:01.692091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87910595-0a0f-4538-8e08-2a2a00740977 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:01.702196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 65/126] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:01.703502 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87910595-0a0f-4538-8e08-2a2a00740977 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:01.710512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3920dc-409e-467f-a4b2-5f45dc864f49 None admin] RBAC: Authorizing `identity:get_user(user_id=abf0cae0793b4db39afb4a7f1e5b6035)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:01.711992 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3920dc-409e-467f-a4b2-5f45dc864f49 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:01.713775 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 62/127] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:01 2022] GET /identity/v3/users/abf0cae0793b4db39afb4a7f1e5b6035 => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:01.715150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3920dc-409e-467f-a4b2-5f45dc864f49 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:01.735034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:01.829027 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87910595-0a0f-4538-8e08-2a2a00740977 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:01.833275 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3920dc-409e-467f-a4b2-5f45dc864f49 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:01.842974 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:01.872329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694363c5-49df-4faa-b4e5-d126c1574c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:01.873463 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694363c5-49df-4faa-b4e5-d126c1574c25 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:01.874062 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694363c5-49df-4faa-b4e5-d126c1574c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:01.874640 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694363c5-49df-4faa-b4e5-d126c1574c25 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:01.875042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ef968-ef68-48de-bfd6-03cb91046127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:01.875695 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ef968-ef68-48de-bfd6-03cb91046127 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:01.875978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ef968-ef68-48de-bfd6-03cb91046127 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:01.876259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ef968-ef68-48de-bfd6-03cb91046127 None admin] PATH_INFO: `/v3/users/b821a3d0f6354acaad7a12c771fb8141` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:01.876358 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694363c5-49df-4faa-b4e5-d126c1574c25 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:01.876912 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694363c5-49df-4faa-b4e5-d126c1574c25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:01.878069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ef968-ef68-48de-bfd6-03cb91046127 None admin] RBAC: Authorizing `identity:get_user(user_id=b821a3d0f6354acaad7a12c771fb8141)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:01.878853 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694363c5-49df-4faa-b4e5-d126c1574c25 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:01.879494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ef968-ef68-48de-bfd6-03cb91046127 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:01.881205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 63/128] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:01 2022] GET /identity/v3/users/b821a3d0f6354acaad7a12c771fb8141 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:01.886354 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ef968-ef68-48de-bfd6-03cb91046127 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:01.889603 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 66/129] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 19:59:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:01.899552 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694363c5-49df-4faa-b4e5-d126c1574c25 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:01.995801 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ef968-ef68-48de-bfd6-03cb91046127 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:01.999092 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:02.017851 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694363c5-49df-4faa-b4e5-d126c1574c25 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:02.047604 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29eb8e-c917-41d0-bb99-91d160b01254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:02.048218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29eb8e-c917-41d0-bb99-91d160b01254 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:02.048516 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29eb8e-c917-41d0-bb99-91d160b01254 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:02.048786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29eb8e-c917-41d0-bb99-91d160b01254 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:02.097059 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:02.107162 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30347b30-c1d1-428c-ac35-7721c039d457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:02.108240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30347b30-c1d1-428c-ac35-7721c039d457 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:02.108731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30347b30-c1d1-428c-ac35-7721c039d457 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:02.109343 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30347b30-c1d1-428c-ac35-7721c039d457 None admin] PATH_INFO: `/v3/users/b821a3d0f6354acaad7a12c771fb8141` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:02.112573 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30347b30-c1d1-428c-ac35-7721c039d457 None admin] RBAC: Authorizing `identity:get_user(user_id=b821a3d0f6354acaad7a12c771fb8141)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:02.114441 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30347b30-c1d1-428c-ac35-7721c039d457 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:02.116613 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 67/130] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:01 2022] GET /identity/v3/users/b821a3d0f6354acaad7a12c771fb8141 => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:02.117625 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30347b30-c1d1-428c-ac35-7721c039d457 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:02.128803 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29eb8e-c917-41d0-bb99-91d160b01254 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:02.130343 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29eb8e-c917-41d0-bb99-91d160b01254 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:02.267552 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30347b30-c1d1-428c-ac35-7721c039d457 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:02.290863 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 64/131] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:02.291234 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29eb8e-c917-41d0-bb99-91d160b01254 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:02.307116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eaf69d-3eca-476a-b27a-861b6ff9c7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:02.308233 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eaf69d-3eca-476a-b27a-861b6ff9c7bc None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:02.308814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eaf69d-3eca-476a-b27a-861b6ff9c7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:02.309318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eaf69d-3eca-476a-b27a-861b6ff9c7bc None admin] PATH_INFO: `/v3/projects/4836001ac84c48758f3fcbff0cd12afc` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:02.310744 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eaf69d-3eca-476a-b27a-861b6ff9c7bc None admin] RBAC: Authorizing `identity:get_project(project_id=4836001ac84c48758f3fcbff0cd12afc)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:02.312394 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eaf69d-3eca-476a-b27a-861b6ff9c7bc None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:02.314284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 68/132] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:02 2022] GET /identity/v3/projects/4836001ac84c48758f3fcbff0cd12afc => generated 312 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:02.315329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eaf69d-3eca-476a-b27a-861b6ff9c7bc None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:02.370181 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:02.431536 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eaf69d-3eca-476a-b27a-861b6ff9c7bc None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:02.436407 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:02.472525 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76559d-3b5f-46b1-8d89-68b5aeeccfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:02.473577 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76559d-3b5f-46b1-8d89-68b5aeeccfeb None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:02.474335 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76559d-3b5f-46b1-8d89-68b5aeeccfeb None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:02.474843 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76559d-3b5f-46b1-8d89-68b5aeeccfeb None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:02.476506 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76559d-3b5f-46b1-8d89-68b5aeeccfeb None admin] RBAC: Authorizing `identity:get_project(project_id=4367ec2ce611409a85975a2a505f1ff8)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:02.478323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76559d-3b5f-46b1-8d89-68b5aeeccfeb None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:02.480484 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 69/133] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:02 2022] GET /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:02.481552 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76559d-3b5f-46b1-8d89-68b5aeeccfeb None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:02.487588 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29eb8e-c917-41d0-bb99-91d160b01254 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:02.561625 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e52ef-ecb8-4a6c-ba5d-a6afc5af899c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:02.562639 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e52ef-ecb8-4a6c-ba5d-a6afc5af899c None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:02.563118 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e52ef-ecb8-4a6c-ba5d-a6afc5af899c None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:02.563587 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e52ef-ecb8-4a6c-ba5d-a6afc5af899c None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:02.566278 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e52ef-ecb8-4a6c-ba5d-a6afc5af899c None admin] RBAC: Authorizing `identity:get_project(project_id=4367ec2ce611409a85975a2a505f1ff8)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:02.568084 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e52ef-ecb8-4a6c-ba5d-a6afc5af899c None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:02.570945 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 65/134] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:02 2022] GET /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8 => generated 298 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:02.577755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e52ef-ecb8-4a6c-ba5d-a6afc5af899c None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:02.672154 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:02.696634 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76559d-3b5f-46b1-8d89-68b5aeeccfeb None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:02.731473 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e52ef-ecb8-4a6c-ba5d-a6afc5af899c None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:02.760765 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e5ab2-8274-415e-8641-8918e6805aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:02.762226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e5ab2-8274-415e-8641-8918e6805aee None admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:02.762509 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e5ab2-8274-415e-8641-8918e6805aee None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:02.762783 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e5ab2-8274-415e-8641-8918e6805aee None admin] PATH_INFO: `/v3/projects/4836001ac84c48758f3fcbff0cd12afc/users/b821a3d0f6354acaad7a12c771fb8141/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:02.765886 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e5ab2-8274-415e-8641-8918e6805aee None admin] RBAC: Authorizing `identity:create_grant(project_id=4836001ac84c48758f3fcbff0cd12afc, user_id=b821a3d0f6354acaad7a12c771fb8141, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:02.767417 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e5ab2-8274-415e-8641-8918e6805aee None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:02.782695 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af40826f-2b14-4101-b8a4-157a3df8da21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:02.783726 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40826f-2b14-4101-b8a4-157a3df8da21 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:02.784236 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40826f-2b14-4101-b8a4-157a3df8da21 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:02.784707 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40826f-2b14-4101-b8a4-157a3df8da21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:02.785790 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af40826f-2b14-4101-b8a4-157a3df8da21 None admin] RBAC: Adding query filter params (role.id=c4b36aca8aba43d09071f52c763a80a8, scope.project.id=4367ec2ce611409a85975a2a505f1ff8, user.id=abf0cae0793b4db39afb4a7f1e5b6035) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:02.786322 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af40826f-2b14-4101-b8a4-157a3df8da21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:02.788122 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af40826f-2b14-4101-b8a4-157a3df8da21 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:02.789819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 70/135] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 19:59:02 2022] PUT /identity/v3/projects/4836001ac84c48758f3fcbff0cd12afc/users/b821a3d0f6354acaad7a12c771fb8141/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:59:02.791154 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e5ab2-8274-415e-8641-8918e6805aee None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:02.796277 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 66/136] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 19:59:02 2022] GET /identity/v3/role_assignments?user.id=abf0cae0793b4db39afb4a7f1e5b6035&scope.project.id=4367ec2ce611409a85975a2a505f1ff8&role.id=c4b36aca8aba43d09071f52c763a80a8 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:02.797794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f4171-78fc-4406-a0d9-bd86e35b8f01 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:02.798286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f4171-78fc-4406-a0d9-bd86e35b8f01 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:02.798783 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f4171-78fc-4406-a0d9-bd86e35b8f01 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:02.800588 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 67/137] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:02.808715 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba1a1d-1c51-4651-ae32-504537ac1814 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:02.809377 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba1a1d-1c51-4651-ae32-504537ac1814 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:02.814105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba1a1d-1c51-4651-ae32-504537ac1814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:02.840444 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:02.856633 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-1cba1a1d-1c51-4651-ae32-504537ac1814 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:02.858009 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-1cba1a1d-1c51-4651-ae32-504537ac1814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:02.998683 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 68/138] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:03.042049 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e5ab2-8274-415e-8641-8918e6805aee None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:03.091400 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08570da-8f40-42b6-8faa-0713df635f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:03.092083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08570da-8f40-42b6-8faa-0713df635f36 None admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:03.092363 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08570da-8f40-42b6-8faa-0713df635f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:03.092640 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08570da-8f40-42b6-8faa-0713df635f36 None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8/users/b821a3d0f6354acaad7a12c771fb8141/roles/b9184599d95b4950beca34304cdd2eb8` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:03.097558 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08570da-8f40-42b6-8faa-0713df635f36 None admin] RBAC: Authorizing `identity:create_grant(project_id=4367ec2ce611409a85975a2a505f1ff8, user_id=b821a3d0f6354acaad7a12c771fb8141, role_id=b9184599d95b4950beca34304cdd2eb8)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:03.099074 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08570da-8f40-42b6-8faa-0713df635f36 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:03.121647 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 71/139] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 19:59:02 2022] PUT /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8/users/b821a3d0f6354acaad7a12c771fb8141/roles/b9184599d95b4950beca34304cdd2eb8 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:59:03.335860 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba1a1d-1c51-4651-ae32-504537ac1814 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:03.450594 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba1a1d-1c51-4651-ae32-504537ac1814 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:03.488174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0faf6-a164-423c-b974-2fb558de6f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:03.488870 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0faf6-a164-423c-b974-2fb558de6f18 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:03.489155 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0faf6-a164-423c-b974-2fb558de6f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:03.489470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0faf6-a164-423c-b974-2fb558de6f18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:03.492384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:03.502829 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d0faf6-a164-423c-b974-2fb558de6f18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:03.507844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-23d0faf6-a164-423c-b974-2fb558de6f18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:03.507844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:03.507844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:03.507844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:03.507844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:03.507844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:03.507844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:03.507844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:03.507844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:03.507844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:03.507844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:03.507844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:03.507844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:03.507844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:03.507844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:03.507844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:03.508673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:03.508673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:03.508673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:03.508673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:03.508673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:03.508673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:03.508673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:03.508673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:03.508673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:03.508673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:03.508673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:03.508673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:03.508673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:03.508673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:03.508673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:03.508673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:03.509878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:03.509878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:03.509878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:03.509878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:03.509878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:03.509878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:03.509878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:03.509878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:03.509878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:03.509878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:03.509878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:03.509878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:03.509878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 69/140] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:03.515897 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08570da-8f40-42b6-8faa-0713df635f36 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:03.537088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:03.661759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08570da-8f40-42b6-8faa-0713df635f36 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:03.710429 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19aaeea-0d18-4c9a-a9a3-fd62b49a21f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:03.711070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19aaeea-0d18-4c9a-a9a3-fd62b49a21f1 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:03.711322 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19aaeea-0d18-4c9a-a9a3-fd62b49a21f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:03.711608 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19aaeea-0d18-4c9a-a9a3-fd62b49a21f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:03.712277 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19aaeea-0d18-4c9a-a9a3-fd62b49a21f1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:03.712575 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19aaeea-0d18-4c9a-a9a3-fd62b49a21f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:03.713986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19aaeea-0d18-4c9a-a9a3-fd62b49a21f1 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:03.722914 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 72/141] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:03.728582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0faf6-a164-423c-b974-2fb558de6f18 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:03.847117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367860b3-c686-49a4-96d7-334232c39b15 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:03.847433 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367860b3-c686-49a4-96d7-334232c39b15 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:03.847672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367860b3-c686-49a4-96d7-334232c39b15 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:03.849261 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 73/142] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:03.854619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cd7b1-680b-4f9c-aaf6-f42fbd870d2f None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:03.855457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cd7b1-680b-4f9c-aaf6-f42fbd870d2f None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:03.855740 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cd7b1-680b-4f9c-aaf6-f42fbd870d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:03.875436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-a85cd7b1-680b-4f9c-aaf6-f42fbd870d2f None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:03.876738 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a85cd7b1-680b-4f9c-aaf6-f42fbd870d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:03.900354 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0faf6-a164-423c-b974-2fb558de6f18 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:03.981136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7e46a-038e-44b0-a733-225e8ad5ab4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:03.981749 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7e46a-038e-44b0-a733-225e8ad5ab4a None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:03.982019 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7e46a-038e-44b0-a733-225e8ad5ab4a None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:03.982299 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7e46a-038e-44b0-a733-225e8ad5ab4a None admin] PATH_INFO: `/v3/users/b821a3d0f6354acaad7a12c771fb8141` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:03.983649 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e7e46a-038e-44b0-a733-225e8ad5ab4a None admin] RBAC: Authorizing `identity:get_user(user_id=b821a3d0f6354acaad7a12c771fb8141)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:03.985124 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e7e46a-038e-44b0-a733-225e8ad5ab4a None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:03.988347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 70/143] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:03 2022] GET /identity/v3/users/b821a3d0f6354acaad7a12c771fb8141 => generated 275 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:03.995657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7e46a-038e-44b0-a733-225e8ad5ab4a None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:04.002109 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:04.009261 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 74/144] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:59:04.102080 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:04.157624 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7e46a-038e-44b0-a733-225e8ad5ab4a None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:04.204335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210c970-5691-4c14-8583-613d14eed93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:04.204967 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210c970-5691-4c14-8583-613d14eed93e None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:04.205264 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210c970-5691-4c14-8583-613d14eed93e None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:04.205537 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210c970-5691-4c14-8583-613d14eed93e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:04.302066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210c970-5691-4c14-8583-613d14eed93e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:04.303465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210c970-5691-4c14-8583-613d14eed93e None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:04.347666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cd7b1-680b-4f9c-aaf6-f42fbd870d2f None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:04.415732 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:04.508572 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 71/145] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 19:59:04.515766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210c970-5691-4c14-8583-613d14eed93e None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:04.554829 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cd7b1-680b-4f9c-aaf6-f42fbd870d2f None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:04.597269 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575173e-6a09-48ce-89f9-c94b90bb01ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:04.597929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575173e-6a09-48ce-89f9-c94b90bb01ef None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:04.598205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575173e-6a09-48ce-89f9-c94b90bb01ef None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:04.598473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575173e-6a09-48ce-89f9-c94b90bb01ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:04.600531 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575173e-6a09-48ce-89f9-c94b90bb01ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:04.602066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575173e-6a09-48ce-89f9-c94b90bb01ef None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:04.606639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 75/146] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:04.606639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d986ca9-3692-4781-b082-72a0b43a024f None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:04.606639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d986ca9-3692-4781-b082-72a0b43a024f None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:04.606639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d986ca9-3692-4781-b082-72a0b43a024f None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:04.611565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 76/147] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:04.613187 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d986ca9-3692-4781-b082-72a0b43a024f None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:04.722993 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210c970-5691-4c14-8583-613d14eed93e None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:04.740498 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:04.787461 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0e362-e285-4068-9c0a-82ef299748a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:04.788114 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0e362-e285-4068-9c0a-82ef299748a3 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:04.788397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0e362-e285-4068-9c0a-82ef299748a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:04.788670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0e362-e285-4068-9c0a-82ef299748a3 None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:04.789968 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0e362-e285-4068-9c0a-82ef299748a3 None admin] RBAC: Authorizing `identity:get_project(project_id=4367ec2ce611409a85975a2a505f1ff8)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:04.791428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0e362-e285-4068-9c0a-82ef299748a3 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:04.793114 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 72/148] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:04 2022] GET /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8 => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:04.797162 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a67de-a5c5-4b28-bf39-4d542e0a4f0f None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:04.797441 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a67de-a5c5-4b28-bf39-4d542e0a4f0f None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:04.798894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a67de-a5c5-4b28-bf39-4d542e0a4f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:04.814917 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d986ca9-3692-4781-b082-72a0b43a024f None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:04.835353 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-566a67de-a5c5-4b28-bf39-4d542e0a4f0f None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:04.836577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-566a67de-a5c5-4b28-bf39-4d542e0a4f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:04.876011 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:04.893380 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b219b2b0-3674-42a7-921b-1264aac59650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:04.894048 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219b2b0-3674-42a7-921b-1264aac59650 None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:04.894466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219b2b0-3674-42a7-921b-1264aac59650 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:04.894810 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219b2b0-3674-42a7-921b-1264aac59650 None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:04.895543 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b219b2b0-3674-42a7-921b-1264aac59650 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:04.897005 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b219b2b0-3674-42a7-921b-1264aac59650 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:04.927914 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 77/149] 10.222.0.73 () {60 vars in 1132 bytes} [Mon Aug 29 19:59:04 2022] POST /identity/v3/users => generated 283 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:59:04.928903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b219b2b0-3674-42a7-921b-1264aac59650 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:04.961930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 73/150] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:05.049527 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b219b2b0-3674-42a7-921b-1264aac59650 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:05.078042 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:05.108301 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01fbf2-9588-4b0a-a082-88561ae707b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:05.109021 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01fbf2-9588-4b0a-a082-88561ae707b0 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:05.109304 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01fbf2-9588-4b0a-a082-88561ae707b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:05.109578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01fbf2-9588-4b0a-a082-88561ae707b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:05.110363 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01fbf2-9588-4b0a-a082-88561ae707b0 None admin] RBAC: Adding query filter params (role.id=9265e1fd617b4c0b8fbf267885394060, scope.project.id=4367ec2ce611409a85975a2a505f1ff8, user.id=b821a3d0f6354acaad7a12c771fb8141) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:05.110677 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01fbf2-9588-4b0a-a082-88561ae707b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:05.112169 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01fbf2-9588-4b0a-a082-88561ae707b0 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:05.118953 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 78/151] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 19:59:04 2022] GET /identity/v3/role_assignments?user.id=b821a3d0f6354acaad7a12c771fb8141&scope.project.id=4367ec2ce611409a85975a2a505f1ff8&role.id=9265e1fd617b4c0b8fbf267885394060 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:05.470041 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a67de-a5c5-4b28-bf39-4d542e0a4f0f None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:05.593383 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a67de-a5c5-4b28-bf39-4d542e0a4f0f None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:05.614366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3562229-dd5b-42b4-b47a-91a1e6925ebe None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:05.614673 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3562229-dd5b-42b4-b47a-91a1e6925ebe None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:05.614927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3562229-dd5b-42b4-b47a-91a1e6925ebe None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:05.616466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 79/152] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:05.623218 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfcf2a-6d8a-4ea5-b0ca-7d2f047ee3f7 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:05.623564 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfcf2a-6d8a-4ea5-b0ca-7d2f047ee3f7 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:05.623843 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfcf2a-6d8a-4ea5-b0ca-7d2f047ee3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:05.639753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4d073-e57e-469a-8af1-09581d742b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:05.640408 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4d073-e57e-469a-8af1-09581d742b55 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:05.640690 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4d073-e57e-469a-8af1-09581d742b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:05.640963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4d073-e57e-469a-8af1-09581d742b55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:05.657310 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-7ddfcf2a-6d8a-4ea5-b0ca-7d2f047ee3f7 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:05.658284 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4d073-e57e-469a-8af1-09581d742b55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:05.658566 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-7ddfcf2a-6d8a-4ea5-b0ca-7d2f047ee3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:05.664497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-61d4d073-e57e-469a-8af1-09581d742b55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:59:05.664497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:05.664497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:05.664497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:05.664497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:05.664497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:05.664497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:05.664497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:05.664497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:05.664497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:05.664497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:05.664497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:05.664497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:05.664497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:05.664497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:05.664497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:05.665324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:05.665324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:05.665324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:05.665324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:05.665324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:05.665324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:05.665324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:05.665324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:05.665324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:05.665324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:05.665324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:05.665324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:05.665324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:05.665324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:05.665324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:05.665324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:05.666101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:05.666101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:05.666101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:05.666101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:05.666101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:05.666101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:05.666101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:05.666101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:05.666101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:05.666101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:05.666101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:59:05.666101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:05.666101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 74/153] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 19:59:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:05.674555 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4d073-e57e-469a-8af1-09581d742b55 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:05.716131 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:05.771731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 80/154] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:05.772924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a61f8-0274-407a-b788-133caa939ffb None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:05.773193 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a61f8-0274-407a-b788-133caa939ffb None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:05.773466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a61f8-0274-407a-b788-133caa939ffb None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:05.774780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 81/155] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:05.780769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce70f8-d605-4d31-93c0-26493047b299 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:05.781037 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce70f8-d605-4d31-93c0-26493047b299 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:05.781336 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce70f8-d605-4d31-93c0-26493047b299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:05.807427 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-26ce70f8-d605-4d31-93c0-26493047b299 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:05.810160 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-26ce70f8-d605-4d31-93c0-26493047b299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:05.831611 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4d073-e57e-469a-8af1-09581d742b55 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:05.872997 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc55105-1a51-4c3f-88d1-1884897e7dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:05.873736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc55105-1a51-4c3f-88d1-1884897e7dbd None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:05.874102 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc55105-1a51-4c3f-88d1-1884897e7dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:05.874451 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc55105-1a51-4c3f-88d1-1884897e7dbd None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:05.875255 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc55105-1a51-4c3f-88d1-1884897e7dbd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:05.875630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc55105-1a51-4c3f-88d1-1884897e7dbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:05.877101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc55105-1a51-4c3f-88d1-1884897e7dbd None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:05.887983 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 75/156] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:05.893543 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc55105-1a51-4c3f-88d1-1884897e7dbd None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:05.904082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 82/157] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:05.970358 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3980a6-5628-46fa-9d23-206acf91b265 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:05.970679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3980a6-5628-46fa-9d23-206acf91b265 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:05.970920 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3980a6-5628-46fa-9d23-206acf91b265 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:05.972282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 83/158] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:59:05.977967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb88b5c-4c9f-42bc-8cf5-0739d76dc71d None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:05.978250 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb88b5c-4c9f-42bc-8cf5-0739d76dc71d None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:05.978525 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb88b5c-4c9f-42bc-8cf5-0739d76dc71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:05.996165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:06.001247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-dcb88b5c-4c9f-42bc-8cf5-0739d76dc71d None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:06.003672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-dcb88b5c-4c9f-42bc-8cf5-0739d76dc71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:06.021449 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc55105-1a51-4c3f-88d1-1884897e7dbd None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:06.066282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:06.090619 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ed2ab-f683-406e-9afe-0f18f383937d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:06.091638 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ed2ab-f683-406e-9afe-0f18f383937d None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:06.092070 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ed2ab-f683-406e-9afe-0f18f383937d None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:06.092353 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ed2ab-f683-406e-9afe-0f18f383937d None admin] PATH_INFO: `/v3/users/7ab6b80a7eaf40e691fcd95e0949b22e` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:06.093615 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ed2ab-f683-406e-9afe-0f18f383937d None admin] RBAC: Authorizing `identity:get_user(user_id=7ab6b80a7eaf40e691fcd95e0949b22e)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:06.100044 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ed2ab-f683-406e-9afe-0f18f383937d None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:06.102082 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 76/159] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:05 2022] GET /identity/v3/users/7ab6b80a7eaf40e691fcd95e0949b22e => generated 247 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:06.112031 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ed2ab-f683-406e-9afe-0f18f383937d None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:06.136650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 84/160] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:59:06.165097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb88b5c-4c9f-42bc-8cf5-0739d76dc71d None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:06.293519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ed2ab-f683-406e-9afe-0f18f383937d None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:06.297958 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb88b5c-4c9f-42bc-8cf5-0739d76dc71d None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:06.348330 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f5289-46a3-4fda-ae2f-961c8da5516a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:06.349656 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f5289-46a3-4fda-ae2f-961c8da5516a None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:06.350274 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f5289-46a3-4fda-ae2f-961c8da5516a None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:06.350843 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f5289-46a3-4fda-ae2f-961c8da5516a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:06.352971 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9ded0-fa0a-45c6-afd4-2595ff895eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:06.353681 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9ded0-fa0a-45c6-afd4-2595ff895eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:06.354066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9ded0-fa0a-45c6-afd4-2595ff895eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:06.354412 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9ded0-fa0a-45c6-afd4-2595ff895eb3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:06.368342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9ded0-fa0a-45c6-afd4-2595ff895eb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:06.374979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-12b9ded0-fa0a-45c6-afd4-2595ff895eb3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 19:59:06.374979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:06.374979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:06.374979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:06.374979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:06.374979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:06.374979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:06.374979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:06.374979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:06.374979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:06.374979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:06.374979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:06.374979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:06.374979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:06.374979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:06.374979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:06.376108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:06.376108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:06.376108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:06.376108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:06.376108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:06.376108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:06.376108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:06.376108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:06.376108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:06.376108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:06.376108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:06.376108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:06.376108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:06.376108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:06.376108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:06.376108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:06.376919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:06.376919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:06.376919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:06.376919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:06.376919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:06.376919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:06.376919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:06.376919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:06.376919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:06.376919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:06.376919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 19:59:06.376919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:06.376919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 85/161] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 19:59:06.377596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9ded0-fa0a-45c6-afd4-2595ff895eb3 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:06.402555 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:06.430091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:06.477614 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f5289-46a3-4fda-ae2f-961c8da5516a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:06.479531 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f5289-46a3-4fda-ae2f-961c8da5516a None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:06.551702 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9ded0-fa0a-45c6-afd4-2595ff895eb3 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:06.619889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a2f60-d315-415c-bcc1-3693e71635f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:06.620627 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a2f60-d315-415c-bcc1-3693e71635f2 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:06.620966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a2f60-d315-415c-bcc1-3693e71635f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:06.621395 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a2f60-d315-415c-bcc1-3693e71635f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:06.648736 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0a2f60-d315-415c-bcc1-3693e71635f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:06.654539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-df0a2f60-d315-415c-bcc1-3693e71635f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:06.654539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:06.654539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:06.654539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:06.654539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:06.654539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:06.654539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:06.654539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:06.654539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:06.654539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:06.654539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:06.654539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:06.654539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:06.654539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:06.654539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:06.654539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:06.655406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:06.655406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:06.655406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:06.655406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:06.655406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:06.655406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:06.655406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:06.655406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:06.655406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:06.655406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:06.655406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:06.655406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:06.655406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:06.655406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:06.655406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:06.655406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:06.656222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:06.656222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:06.656222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:06.656222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:06.656222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:06.656222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:06.656222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:06.656222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:06.656222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:06.656222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:06.656222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:06.656222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:06.656222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 86/162] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:06.656956 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a2f60-d315-415c-bcc1-3693e71635f2 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:06.667653 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 77/163] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:06.668986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f5289-46a3-4fda-ae2f-961c8da5516a None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:06.729289 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:06.786256 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f5289-46a3-4fda-ae2f-961c8da5516a None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:06.843497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aed38d-55bc-4fe8-9df7-56f83a61ecd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:06.844161 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aed38d-55bc-4fe8-9df7-56f83a61ecd3 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:06.844444 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aed38d-55bc-4fe8-9df7-56f83a61ecd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:06.844721 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aed38d-55bc-4fe8-9df7-56f83a61ecd3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:06.856434 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a2f60-d315-415c-bcc1-3693e71635f2 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:06.867288 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:06.882742 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aed38d-55bc-4fe8-9df7-56f83a61ecd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:06.887931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-68aed38d-55bc-4fe8-9df7-56f83a61ecd3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:06.887931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:06.887931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:06.887931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:06.887931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:06.887931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:06.887931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:06.887931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:06.887931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:06.887931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:06.887931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:06.887931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:06.887931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:06.887931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:06.887931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:06.887931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:06.889061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:06.889061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:06.889061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:06.889061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:06.889061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:06.889061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:06.889061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:06.889061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:06.889061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:06.889061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:06.889061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:06.889061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:06.889061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:06.889061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:06.889061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:06.889061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:06.889843 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:06.889843 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:06.889843 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:06.889843 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:06.889843 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:06.889843 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:06.889843 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:06.889843 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:06.889843 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:06.889843 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:06.889843 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:06.889843 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:06.889843 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 78/164] 10.222.0.73 () {56 vars in 1114 bytes} [Mon Aug 29 19:59:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:06.890488 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aed38d-55bc-4fe8-9df7-56f83a61ecd3 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:06.903935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09a7f3-7bd6-4111-95db-b4b175c49557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:06.904611 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09a7f3-7bd6-4111-95db-b4b175c49557 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:06.905093 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09a7f3-7bd6-4111-95db-b4b175c49557 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:06.905369 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09a7f3-7bd6-4111-95db-b4b175c49557 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:06.907077 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09a7f3-7bd6-4111-95db-b4b175c49557 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:06.907409 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09a7f3-7bd6-4111-95db-b4b175c49557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:06.908838 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09a7f3-7bd6-4111-95db-b4b175c49557 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:06.919210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 87/165] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:06.920564 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09a7f3-7bd6-4111-95db-b4b175c49557 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:07.097732 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aed38d-55bc-4fe8-9df7-56f83a61ecd3 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:07.098204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09a7f3-7bd6-4111-95db-b4b175c49557 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:07.114110 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:07.148288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1d184-1fa6-422b-a29d-5c145b41baf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:07.148830 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1d184-1fa6-422b-a29d-5c145b41baf5 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:07.149109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1d184-1fa6-422b-a29d-5c145b41baf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:07.149379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1d184-1fa6-422b-a29d-5c145b41baf5 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:07.150078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb1d184-1fa6-422b-a29d-5c145b41baf5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:07.150379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb1d184-1fa6-422b-a29d-5c145b41baf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:07.151824 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb1d184-1fa6-422b-a29d-5c145b41baf5 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:07.171997 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 79/166] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:07.172846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1d184-1fa6-422b-a29d-5c145b41baf5 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:07.175222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b4f12-98ca-425b-ae3b-6e7bbaf5846d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:07.175861 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b4f12-98ca-425b-ae3b-6e7bbaf5846d None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:07.176138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b4f12-98ca-425b-ae3b-6e7bbaf5846d None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:07.176409 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b4f12-98ca-425b-ae3b-6e7bbaf5846d None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:07.177994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b4f12-98ca-425b-ae3b-6e7bbaf5846d None admin] RBAC: Authorizing `identity:get_project(project_id=4367ec2ce611409a85975a2a505f1ff8)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:07.179506 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b4f12-98ca-425b-ae3b-6e7bbaf5846d None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:07.181150 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 88/167] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:06 2022] GET /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:07.182033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b4f12-98ca-425b-ae3b-6e7bbaf5846d None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:07.223377 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:07.327415 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b4f12-98ca-425b-ae3b-6e7bbaf5846d None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:07.390313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1d184-1fa6-422b-a29d-5c145b41baf5 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:07.395678 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356def8-cd7a-4256-84f1-2f8eb0ef975c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:07.396466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356def8-cd7a-4256-84f1-2f8eb0ef975c None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:07.396742 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356def8-cd7a-4256-84f1-2f8eb0ef975c None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:07.397013 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356def8-cd7a-4256-84f1-2f8eb0ef975c None admin] PATH_INFO: `/v3/users/abf0cae0793b4db39afb4a7f1e5b6035` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:07.402532 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356def8-cd7a-4256-84f1-2f8eb0ef975c None admin] RBAC: Authorizing `identity:get_user(user_id=abf0cae0793b4db39afb4a7f1e5b6035)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:07.403951 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356def8-cd7a-4256-84f1-2f8eb0ef975c None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:07.405617 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 89/168] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:07 2022] GET /identity/v3/users/abf0cae0793b4db39afb4a7f1e5b6035 => generated 289 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:07.406688 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356def8-cd7a-4256-84f1-2f8eb0ef975c None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:07.451083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df111e7a-613d-495d-b2a3-2efd43557ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:07.451713 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df111e7a-613d-495d-b2a3-2efd43557ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:07.452000 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df111e7a-613d-495d-b2a3-2efd43557ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:07.452284 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df111e7a-613d-495d-b2a3-2efd43557ae9 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:07.452972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df111e7a-613d-495d-b2a3-2efd43557ae9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:07.453282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df111e7a-613d-495d-b2a3-2efd43557ae9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:07.454725 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df111e7a-613d-495d-b2a3-2efd43557ae9 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:07.464513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 80/169] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 19:59:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:07.466504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df111e7a-613d-495d-b2a3-2efd43557ae9 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:07.549387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:07.572100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:07.580473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356def8-cd7a-4256-84f1-2f8eb0ef975c None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:07.644402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bce90-8854-448f-a868-2f88de98955d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:07.645508 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bce90-8854-448f-a868-2f88de98955d None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:07.646012 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bce90-8854-448f-a868-2f88de98955d None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:07.646501 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bce90-8854-448f-a868-2f88de98955d None admin] PATH_INFO: `/v3/users/b821a3d0f6354acaad7a12c771fb8141` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:07.648659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7bce90-8854-448f-a868-2f88de98955d None admin] RBAC: Authorizing `identity:get_user(user_id=b821a3d0f6354acaad7a12c771fb8141)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:07.650391 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7bce90-8854-448f-a868-2f88de98955d None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:07.655831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 90/170] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:07 2022] GET /identity/v3/users/b821a3d0f6354acaad7a12c771fb8141 => generated 275 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:07.656950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bce90-8854-448f-a868-2f88de98955d None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:07.696367 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df111e7a-613d-495d-b2a3-2efd43557ae9 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:07.741081 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115de42-4fd8-4813-b48e-fe942a54646c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:07.742427 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115de42-4fd8-4813-b48e-fe942a54646c None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:07.743136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115de42-4fd8-4813-b48e-fe942a54646c None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:07.743845 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115de42-4fd8-4813-b48e-fe942a54646c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:07.745172 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115de42-4fd8-4813-b48e-fe942a54646c None admin] RBAC: Adding query filter params (role.id=296ee2c194824485af71618619f82e09, scope.project.id=4367ec2ce611409a85975a2a505f1ff8, user.id=7ab6b80a7eaf40e691fcd95e0949b22e) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:07.746417 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115de42-4fd8-4813-b48e-fe942a54646c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:07.748103 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115de42-4fd8-4813-b48e-fe942a54646c None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:07.754831 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 81/171] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 19:59:07 2022] GET /identity/v3/role_assignments?user.id=7ab6b80a7eaf40e691fcd95e0949b22e&scope.project.id=4367ec2ce611409a85975a2a505f1ff8&role.id=296ee2c194824485af71618619f82e09 => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:07.756072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115de42-4fd8-4813-b48e-fe942a54646c None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:07.830097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bce90-8854-448f-a868-2f88de98955d None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:07.910511 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84ec8f-d0b9-4928-bfc1-88728d39884a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:07.911107 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84ec8f-d0b9-4928-bfc1-88728d39884a None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:07.911370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84ec8f-d0b9-4928-bfc1-88728d39884a None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:07.911596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84ec8f-d0b9-4928-bfc1-88728d39884a None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:07.915093 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd84ec8f-d0b9-4928-bfc1-88728d39884a None admin] RBAC: Authorizing `identity:get_project(project_id=4367ec2ce611409a85975a2a505f1ff8)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:07.918439 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd84ec8f-d0b9-4928-bfc1-88728d39884a None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:07.920400 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 91/172] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:07 2022] GET /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:07.921198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84ec8f-d0b9-4928-bfc1-88728d39884a None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:07.949972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:07.953839 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115de42-4fd8-4813-b48e-fe942a54646c None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:08.000472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:08.032476 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ddd53-34ef-46c9-829e-c961cb4d79cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:08.033174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ddd53-34ef-46c9-829e-c961cb4d79cc None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:08.033450 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ddd53-34ef-46c9-829e-c961cb4d79cc None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:08.033967 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ddd53-34ef-46c9-829e-c961cb4d79cc None admin] PATH_INFO: `/v3/users/b821a3d0f6354acaad7a12c771fb8141` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:08.036500 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ddd53-34ef-46c9-829e-c961cb4d79cc None admin] RBAC: Authorizing `identity:get_user(user_id=b821a3d0f6354acaad7a12c771fb8141)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:08.038928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ddd53-34ef-46c9-829e-c961cb4d79cc None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:08.048017 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 82/173] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:07 2022] GET /identity/v3/users/b821a3d0f6354acaad7a12c771fb8141 => generated 275 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:08.049476 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ddd53-34ef-46c9-829e-c961cb4d79cc None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:08.105930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84ec8f-d0b9-4928-bfc1-88728d39884a None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:08.147380 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256c1e7-341d-4b80-a19d-8b77fb9c34fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:08.148186 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256c1e7-341d-4b80-a19d-8b77fb9c34fd None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:08.148540 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256c1e7-341d-4b80-a19d-8b77fb9c34fd None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:08.148889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256c1e7-341d-4b80-a19d-8b77fb9c34fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:08.171476 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ddd53-34ef-46c9-829e-c961cb4d79cc None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:08.209670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4603f-0e96-4a70-8cce-384931acb9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:08.210390 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4603f-0e96-4a70-8cce-384931acb9cc None admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:08.210671 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4603f-0e96-4a70-8cce-384931acb9cc None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:08.210944 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4603f-0e96-4a70-8cce-384931acb9cc None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8/users/abf0cae0793b4db39afb4a7f1e5b6035/roles/c4b36aca8aba43d09071f52c763a80a8` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:08.212929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b4603f-0e96-4a70-8cce-384931acb9cc None admin] RBAC: Authorizing `identity:create_grant(project_id=4367ec2ce611409a85975a2a505f1ff8, user_id=abf0cae0793b4db39afb4a7f1e5b6035, role_id=c4b36aca8aba43d09071f52c763a80a8)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:08.214706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b4603f-0e96-4a70-8cce-384931acb9cc None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:08.229498 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 83/174] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 19:59:08 2022] PUT /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8/users/abf0cae0793b4db39afb4a7f1e5b6035/roles/c4b36aca8aba43d09071f52c763a80a8 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:59:08.231048 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4603f-0e96-4a70-8cce-384931acb9cc None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:08.237351 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256c1e7-341d-4b80-a19d-8b77fb9c34fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:08.239244 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256c1e7-341d-4b80-a19d-8b77fb9c34fd None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:08.266389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:08.294851 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:08.427869 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4603f-0e96-4a70-8cce-384931acb9cc None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:08.447844 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 92/175] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:08.451551 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa116787-f178-4501-a036-aa0b7cacaa42 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:08.452225 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa116787-f178-4501-a036-aa0b7cacaa42 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:08.453597 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa116787-f178-4501-a036-aa0b7cacaa42 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:08.457818 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 93/176] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:08.458621 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa116787-f178-4501-a036-aa0b7cacaa42 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:08.480631 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b0968-0c65-4aa7-9f89-b0659f1c5339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:08.481737 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b0968-0c65-4aa7-9f89-b0659f1c5339 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:08.482253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b0968-0c65-4aa7-9f89-b0659f1c5339 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:08.482719 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b0968-0c65-4aa7-9f89-b0659f1c5339 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:08.578183 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b0968-0c65-4aa7-9f89-b0659f1c5339 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:08.579976 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b0968-0c65-4aa7-9f89-b0659f1c5339 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:08.614408 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa116787-f178-4501-a036-aa0b7cacaa42 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:08.616940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:08.670106 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:08.679326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc1fe8-02f4-4466-9dc2-bccfeecdeaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:08.679893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc1fe8-02f4-4466-9dc2-bccfeecdeaa4 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:08.680171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc1fe8-02f4-4466-9dc2-bccfeecdeaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:08.680446 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc1fe8-02f4-4466-9dc2-bccfeecdeaa4 None admin] PATH_INFO: `/v3/projects/4836001ac84c48758f3fcbff0cd12afc` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:08.681677 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc1fe8-02f4-4466-9dc2-bccfeecdeaa4 None admin] RBAC: Authorizing `identity:get_project(project_id=4836001ac84c48758f3fcbff0cd12afc)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:08.683190 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc1fe8-02f4-4466-9dc2-bccfeecdeaa4 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:08.685087 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 94/177] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:08 2022] GET /identity/v3/projects/4836001ac84c48758f3fcbff0cd12afc => generated 312 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:08.689922 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938c99c-8ef9-40df-9e0d-8cfd7b9c1ed4 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:08.690204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938c99c-8ef9-40df-9e0d-8cfd7b9c1ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:08.690476 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938c99c-8ef9-40df-9e0d-8cfd7b9c1ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:08.728336 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-7938c99c-8ef9-40df-9e0d-8cfd7b9c1ed4 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:08.729553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-7938c99c-8ef9-40df-9e0d-8cfd7b9c1ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:08.769401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 84/178] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:08.770512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b0968-0c65-4aa7-9f89-b0659f1c5339 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:08.825399 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 95/179] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:08.831766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142f2e2-d605-4ddd-aa7a-7700aca814be None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:08.832115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142f2e2-d605-4ddd-aa7a-7700aca814be None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:08.832363 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142f2e2-d605-4ddd-aa7a-7700aca814be None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:08.834029 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 96/180] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:08.835426 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e02a4a-73ae-4f24-ae3e-9045e3f6cc23 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:08.835710 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e02a4a-73ae-4f24-ae3e-9045e3f6cc23 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:08.835991 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e02a4a-73ae-4f24-ae3e-9045e3f6cc23 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:08.837997 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 97/181] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:08.842004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e02a4a-73ae-4f24-ae3e-9045e3f6cc23 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:08.896487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b0968-0c65-4aa7-9f89-b0659f1c5339 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:08.931177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:08.942269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c3c9c-ac84-4439-9398-ffc7823d13a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:08.943235 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c3c9c-ac84-4439-9398-ffc7823d13a8 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:08.943519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c3c9c-ac84-4439-9398-ffc7823d13a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:08.943786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c3c9c-ac84-4439-9398-ffc7823d13a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:08.944833 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c3c9c-ac84-4439-9398-ffc7823d13a8 None admin] RBAC: Adding query filter params (role.id=9265e1fd617b4c0b8fbf267885394060, scope.project.id=4836001ac84c48758f3fcbff0cd12afc, user.id=b821a3d0f6354acaad7a12c771fb8141) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:08.945140 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c3c9c-ac84-4439-9398-ffc7823d13a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:08.946880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c3c9c-ac84-4439-9398-ffc7823d13a8 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:08.955260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 85/182] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 19:59:08 2022] GET /identity/v3/role_assignments?user.id=b821a3d0f6354acaad7a12c771fb8141&scope.project.id=4836001ac84c48758f3fcbff0cd12afc&role.id=9265e1fd617b4c0b8fbf267885394060 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:08.956907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e132a7-f785-4cfe-b43d-0ef6bd2919a2 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:08.957462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e132a7-f785-4cfe-b43d-0ef6bd2919a2 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:08.957768 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e132a7-f785-4cfe-b43d-0ef6bd2919a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:08.971856 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e02a4a-73ae-4f24-ae3e-9045e3f6cc23 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:08.980686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-c9e132a7-f785-4cfe-b43d-0ef6bd2919a2 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:08.982543 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c9e132a7-f785-4cfe-b43d-0ef6bd2919a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:09.015359 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:09.024366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d867077-e4d9-4845-8a9f-3a309c0f4390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:09.025369 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d867077-e4d9-4845-8a9f-3a309c0f4390 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:09.025652 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d867077-e4d9-4845-8a9f-3a309c0f4390 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:09.025957 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d867077-e4d9-4845-8a9f-3a309c0f4390 None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:09.028283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d867077-e4d9-4845-8a9f-3a309c0f4390 None admin] RBAC: Authorizing `identity:get_project(project_id=4367ec2ce611409a85975a2a505f1ff8)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:09.029947 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d867077-e4d9-4845-8a9f-3a309c0f4390 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:09.031884 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 98/183] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:08 2022] GET /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:09.033271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2288f-06c4-4607-80c0-0128c4ee9e0e None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:09.040950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2288f-06c4-4607-80c0-0128c4ee9e0e None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:09.040950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2288f-06c4-4607-80c0-0128c4ee9e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:09.056001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-30d2288f-06c4-4607-80c0-0128c4ee9e0e None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:09.058556 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-30d2288f-06c4-4607-80c0-0128c4ee9e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:09.114517 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 86/184] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:09.117297 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e132a7-f785-4cfe-b43d-0ef6bd2919a2 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:09.183898 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 99/185] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:09.184652 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2288f-06c4-4607-80c0-0128c4ee9e0e None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:09.319166 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e132a7-f785-4cfe-b43d-0ef6bd2919a2 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:09.338215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2288f-06c4-4607-80c0-0128c4ee9e0e None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:09.346526 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:09.365664 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f45d7-8d49-4c01-9317-a7f873a1bb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:09.366310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f45d7-8d49-4c01-9317-a7f873a1bb74 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:09.366591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f45d7-8d49-4c01-9317-a7f873a1bb74 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:09.366867 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f45d7-8d49-4c01-9317-a7f873a1bb74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:09.367605 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f45d7-8d49-4c01-9317-a7f873a1bb74 None admin] RBAC: Adding query filter params (role.id=b9184599d95b4950beca34304cdd2eb8, scope.project.id=4367ec2ce611409a85975a2a505f1ff8, user.id=b821a3d0f6354acaad7a12c771fb8141) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:09.367960 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f45d7-8d49-4c01-9317-a7f873a1bb74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:09.369628 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f45d7-8d49-4c01-9317-a7f873a1bb74 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:09.386156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 87/186] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 19:59:09 2022] GET /identity/v3/role_assignments?user.id=b821a3d0f6354acaad7a12c771fb8141&scope.project.id=4367ec2ce611409a85975a2a505f1ff8&role.id=b9184599d95b4950beca34304cdd2eb8 => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:09.406942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:09.435583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da386041-7549-403a-b086-1a944f094794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:09.436157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da386041-7549-403a-b086-1a944f094794 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:09.436440 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da386041-7549-403a-b086-1a944f094794 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:09.436717 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da386041-7549-403a-b086-1a944f094794 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:09.462927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da386041-7549-403a-b086-1a944f094794 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:09.468133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-da386041-7549-403a-b086-1a944f094794 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:59:09.468133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:09.468133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:09.468133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:09.468133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:09.468133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:09.468133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:09.468133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:09.468133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:09.468133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:09.468133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:09.468133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:09.468133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:09.468133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:09.468133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:09.468133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:09.469086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:09.469086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:09.469086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:09.469086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:09.469086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:09.469086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:09.469086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:09.469086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:09.469086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:09.469086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:09.469086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:09.469086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:09.469086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:09.469086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:09.469086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:09.469086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:09.471467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:09.471467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:09.471467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:09.471467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:09.471467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:09.471467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:09.471467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:09.471467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:09.471467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:09.471467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:09.471467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:59:09.471467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:09.471467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 100/187] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 19:59:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:09.475693 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f45d7-8d49-4c01-9317-a7f873a1bb74 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:09.477900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da386041-7549-403a-b086-1a944f094794 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:09.601586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da386041-7549-403a-b086-1a944f094794 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:09.654313 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d8346-9f4c-4d2b-904a-406728eaa890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:09.654971 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d8346-9f4c-4d2b-904a-406728eaa890 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:09.655139 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f45d7-8d49-4c01-9317-a7f873a1bb74 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:09.655303 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d8346-9f4c-4d2b-904a-406728eaa890 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:09.655508 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d8346-9f4c-4d2b-904a-406728eaa890 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:09.669869 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d8346-9f4c-4d2b-904a-406728eaa890 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:09.675153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-a18d8346-9f4c-4d2b-904a-406728eaa890 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:09.675153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:09.675153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:09.675153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:09.675153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:09.675153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:09.675153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:09.675153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:09.675153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:09.675153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:09.675153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:09.675153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:09.675153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:09.675153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:09.675153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:09.675153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:09.676044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:09.676044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:09.676044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:09.676044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:09.676044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:09.676044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:09.676044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:09.676044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:09.676044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:09.676044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:09.676044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:09.676044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:09.676044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:09.676044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:09.676044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:09.676044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:09.676842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:09.676842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:09.676842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:09.676842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:09.676842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:09.676842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:09.676842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:09.676842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:09.676842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:09.676842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:09.676842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:09.676842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:09.676842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 101/188] 10.222.0.73 () {56 vars in 1114 bytes} [Mon Aug 29 19:59:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 19:59:09.681887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d8346-9f4c-4d2b-904a-406728eaa890 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:09.703941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f63881-472d-489e-856e-912fa679c01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:09.704951 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f63881-472d-489e-856e-912fa679c01b None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:09.705681 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f63881-472d-489e-856e-912fa679c01b None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:09.706000 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f63881-472d-489e-856e-912fa679c01b None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:09.706690 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f63881-472d-489e-856e-912fa679c01b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:09.706999 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f63881-472d-489e-856e-912fa679c01b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:09.708852 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f63881-472d-489e-856e-912fa679c01b None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:09.719268 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 88/189] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:09.719930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f63881-472d-489e-856e-912fa679c01b None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:09.740918 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:09.780740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:09.797632 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d8346-9f4c-4d2b-904a-406728eaa890 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:09.835250 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20907140-21d5-4847-bba5-af79454df2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:09.835893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20907140-21d5-4847-bba5-af79454df2e0 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:09.836167 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20907140-21d5-4847-bba5-af79454df2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:09.836437 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20907140-21d5-4847-bba5-af79454df2e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:09.837560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20907140-21d5-4847-bba5-af79454df2e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:09.839107 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20907140-21d5-4847-bba5-af79454df2e0 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:09.841545 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 102/190] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:09.842328 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20907140-21d5-4847-bba5-af79454df2e0 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:09.845998 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f63881-472d-489e-856e-912fa679c01b None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:09.885542 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96bea4-cb08-469e-8aac-00ac62642fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:09.886332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96bea4-cb08-469e-8aac-00ac62642fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:09.886624 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96bea4-cb08-469e-8aac-00ac62642fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:09.886895 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96bea4-cb08-469e-8aac-00ac62642fa0 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:09.887567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96bea4-cb08-469e-8aac-00ac62642fa0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:09.887985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96bea4-cb08-469e-8aac-00ac62642fa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:09.889479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96bea4-cb08-469e-8aac-00ac62642fa0 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:09.898741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 89/191] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 19:59:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:09.904390 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96bea4-cb08-469e-8aac-00ac62642fa0 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:09.952185 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20907140-21d5-4847-bba5-af79454df2e0 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:09.969392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:09.990103 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce356fa-1b6b-43d8-a4e7-f77db54001ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:09.990848 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce356fa-1b6b-43d8-a4e7-f77db54001ca None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:09.991132 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce356fa-1b6b-43d8-a4e7-f77db54001ca None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:09.991407 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce356fa-1b6b-43d8-a4e7-f77db54001ca None admin] PATH_INFO: `/v3/users/217929a84219411991de135622e7c036` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:10.006384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce356fa-1b6b-43d8-a4e7-f77db54001ca None admin] RBAC: Authorizing `identity:get_user(user_id=217929a84219411991de135622e7c036)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:10.007840 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce356fa-1b6b-43d8-a4e7-f77db54001ca None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:10.009053 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96bea4-cb08-469e-8aac-00ac62642fa0 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:10.009513 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 103/192] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:09 2022] GET /identity/v3/users/217929a84219411991de135622e7c036 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:10.010312 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce356fa-1b6b-43d8-a4e7-f77db54001ca None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:10.025123 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:10.054351 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe82c8-68bc-4ffd-aa37-4b605611e49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:10.055023 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe82c8-68bc-4ffd-aa37-4b605611e49c None admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:10.055306 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe82c8-68bc-4ffd-aa37-4b605611e49c None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:10.055577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe82c8-68bc-4ffd-aa37-4b605611e49c None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:10.056336 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe82c8-68bc-4ffd-aa37-4b605611e49c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:10.057779 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe82c8-68bc-4ffd-aa37-4b605611e49c None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:10.082142 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 90/193] 10.222.0.73 () {60 vars in 1132 bytes} [Mon Aug 29 19:59:09 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:59:10.095304 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe82c8-68bc-4ffd-aa37-4b605611e49c None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:10.139263 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce356fa-1b6b-43d8-a4e7-f77db54001ca None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:10.185783 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449cacb-a6c0-4ae5-a187-a29746aa6f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:10.186780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449cacb-a6c0-4ae5-a187-a29746aa6f83 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:10.186780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449cacb-a6c0-4ae5-a187-a29746aa6f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:10.187053 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449cacb-a6c0-4ae5-a187-a29746aa6f83 None admin] PATH_INFO: `/v3/users/217929a84219411991de135622e7c036` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:10.188223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449cacb-a6c0-4ae5-a187-a29746aa6f83 None admin] RBAC: Authorizing `identity:get_user(user_id=217929a84219411991de135622e7c036)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:10.190000 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449cacb-a6c0-4ae5-a187-a29746aa6f83 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:10.191762 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 104/194] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:10 2022] GET /identity/v3/users/217929a84219411991de135622e7c036 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:10.197836 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449cacb-a6c0-4ae5-a187-a29746aa6f83 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:10.257556 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe82c8-68bc-4ffd-aa37-4b605611e49c None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:10.260200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:10.307705 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3e1d3-3447-4afd-a0b6-a39c964feda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:10.309665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3e1d3-3447-4afd-a0b6-a39c964feda1 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:10.310134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3e1d3-3447-4afd-a0b6-a39c964feda1 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:10.310550 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3e1d3-3447-4afd-a0b6-a39c964feda1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:10.342252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449cacb-a6c0-4ae5-a187-a29746aa6f83 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:10.359527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:10.395129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63e8a0-ef14-4d7c-8277-a7e7efd0f9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:10.395749 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63e8a0-ef14-4d7c-8277-a7e7efd0f9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:10.396029 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63e8a0-ef14-4d7c-8277-a7e7efd0f9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:10.396302 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63e8a0-ef14-4d7c-8277-a7e7efd0f9b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:10.405267 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3e1d3-3447-4afd-a0b6-a39c964feda1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:10.406802 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3e1d3-3447-4afd-a0b6-a39c964feda1 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:10.524655 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63e8a0-ef14-4d7c-8277-a7e7efd0f9b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:10.529186 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63e8a0-ef14-4d7c-8277-a7e7efd0f9b5 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:10.535142 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 91/195] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:10.549268 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3e1d3-3447-4afd-a0b6-a39c964feda1 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:10.592195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:10.628864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:10.667966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 105/196] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 19:59:10.667966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c75b6-634c-4aa1-97d1-533a6726baf1 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:10.667966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c75b6-634c-4aa1-97d1-533a6726baf1 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:10.667966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c75b6-634c-4aa1-97d1-533a6726baf1 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:10.668370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 106/197] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:10.671215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3e1d3-3447-4afd-a0b6-a39c964feda1 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:10.674581 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0de30-14c1-4b86-9b9c-14177ee389ad None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:10.674935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0de30-14c1-4b86-9b9c-14177ee389ad None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:10.675370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0de30-14c1-4b86-9b9c-14177ee389ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:10.700578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-c8c0de30-14c1-4b86-9b9c-14177ee389ad None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:10.702079 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c8c0de30-14c1-4b86-9b9c-14177ee389ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:10.710675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0421b-28be-4f90-944e-142641e77578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:10.711632 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0421b-28be-4f90-944e-142641e77578 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:10.712093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0421b-28be-4f90-944e-142641e77578 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:10.712560 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0421b-28be-4f90-944e-142641e77578 None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:10.713937 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0421b-28be-4f90-944e-142641e77578 None admin] RBAC: Authorizing `identity:get_project(project_id=2e1ab10529d34aaabbf5bdd2cff84222)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:10.716324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0421b-28be-4f90-944e-142641e77578 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:10.718397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 92/198] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:10 2022] GET /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:10.723252 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0421b-28be-4f90-944e-142641e77578 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:10.809829 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 107/199] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:10.811149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0de30-14c1-4b86-9b9c-14177ee389ad None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:10.859173 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0421b-28be-4f90-944e-142641e77578 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:10.915529 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36ecb7-7c74-467d-ba3f-b5fd1f9518ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:10.916179 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36ecb7-7c74-467d-ba3f-b5fd1f9518ad None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:10.916461 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36ecb7-7c74-467d-ba3f-b5fd1f9518ad None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:10.916736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36ecb7-7c74-467d-ba3f-b5fd1f9518ad None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:10.918030 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36ecb7-7c74-467d-ba3f-b5fd1f9518ad None admin] RBAC: Authorizing `identity:get_project(project_id=2e1ab10529d34aaabbf5bdd2cff84222)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:10.919733 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36ecb7-7c74-467d-ba3f-b5fd1f9518ad None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:10.926519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 93/200] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:10 2022] GET /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:10.928255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:10.928662 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36ecb7-7c74-467d-ba3f-b5fd1f9518ad None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:10.935334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:10.967809 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0de30-14c1-4b86-9b9c-14177ee389ad None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:11.034497 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ae6fb-0970-44be-98f3-851a405d6dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:11.035112 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ae6fb-0970-44be-98f3-851a405d6dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:11.035396 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ae6fb-0970-44be-98f3-851a405d6dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:11.035667 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ae6fb-0970-44be-98f3-851a405d6dc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:11.036400 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02ae6fb-0970-44be-98f3-851a405d6dc4 None admin] RBAC: Adding query filter params (role.id=296ee2c194824485af71618619f82e09, scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222, user.id=217929a84219411991de135622e7c036) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:11.036707 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02ae6fb-0970-44be-98f3-851a405d6dc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:11.038184 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02ae6fb-0970-44be-98f3-851a405d6dc4 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:11.051778 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 108/201] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 19:59:10 2022] GET /identity/v3/role_assignments?user.id=217929a84219411991de135622e7c036&scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222&role.id=296ee2c194824485af71618619f82e09 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:11.088641 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36ecb7-7c74-467d-ba3f-b5fd1f9518ad None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:11.107307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ae6fb-0970-44be-98f3-851a405d6dc4 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:11.168857 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb7c94-2ae3-4494-bd38-33b97eeeca56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:11.169483 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb7c94-2ae3-4494-bd38-33b97eeeca56 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:11.169797 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb7c94-2ae3-4494-bd38-33b97eeeca56 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:11.170074 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb7c94-2ae3-4494-bd38-33b97eeeca56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:11.170999 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb7c94-2ae3-4494-bd38-33b97eeeca56 None admin] RBAC: Adding query filter params (role.id=b9184599d95b4950beca34304cdd2eb8, scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222, user.id=217929a84219411991de135622e7c036) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:11.171313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb7c94-2ae3-4494-bd38-33b97eeeca56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:11.173107 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb7c94-2ae3-4494-bd38-33b97eeeca56 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:11.182658 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 94/202] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 19:59:10 2022] GET /identity/v3/role_assignments?user.id=217929a84219411991de135622e7c036&scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222&role.id=b9184599d95b4950beca34304cdd2eb8 => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:11.273380 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ae6fb-0970-44be-98f3-851a405d6dc4 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:11.316259 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef1277-f65a-46c0-b7f8-6103d3ec3787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:11.316981 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef1277-f65a-46c0-b7f8-6103d3ec3787 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:11.317262 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef1277-f65a-46c0-b7f8-6103d3ec3787 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:11.317539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef1277-f65a-46c0-b7f8-6103d3ec3787 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:11.320707 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:11.336948 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef1277-f65a-46c0-b7f8-6103d3ec3787 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:11.343873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-05ef1277-f65a-46c0-b7f8-6103d3ec3787 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 19:59:11.343873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:11.343873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:11.343873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:11.343873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:11.343873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:11.343873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:11.343873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:11.343873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:11.343873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:11.343873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:11.343873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:11.343873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:11.343873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:11.343873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:11.343873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:11.344799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:11.344799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:11.344799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:11.344799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:11.344799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:11.344799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:11.344799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:11.344799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:11.344799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:11.344799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:11.344799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:11.344799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:11.344799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:11.344799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:11.344799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:11.344799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:11.345591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:11.345591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:11.345591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:11.345591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:11.345591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:11.345591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:11.345591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:11.345591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:11.345591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:11.345591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:11.345591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 19:59:11.345591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:11.345591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 109/203] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 19:59:11.352865 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb7c94-2ae3-4494-bd38-33b97eeeca56 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:11.428885 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:11.467025 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb7c94-2ae3-4494-bd38-33b97eeeca56 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:11.502814 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c367f-bd61-4ed9-a6e9-37f42f69c771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:11.503431 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c367f-bd61-4ed9-a6e9-37f42f69c771 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:11.503714 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c367f-bd61-4ed9-a6e9-37f42f69c771 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:11.503988 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c367f-bd61-4ed9-a6e9-37f42f69c771 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:11.504667 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c367f-bd61-4ed9-a6e9-37f42f69c771 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:11.504970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c367f-bd61-4ed9-a6e9-37f42f69c771 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:11.506376 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c367f-bd61-4ed9-a6e9-37f42f69c771 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:11.516055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 95/204] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:11.521836 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef1277-f65a-46c0-b7f8-6103d3ec3787 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:11.659449 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef1277-f65a-46c0-b7f8-6103d3ec3787 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:11.709831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9190601-cb0a-4276-935e-6547159741d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:11.710476 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9190601-cb0a-4276-935e-6547159741d7 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:11.710755 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9190601-cb0a-4276-935e-6547159741d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:11.711030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9190601-cb0a-4276-935e-6547159741d7 None admin] PATH_INFO: `/v3/users/abf0cae0793b4db39afb4a7f1e5b6035` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:11.714721 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9190601-cb0a-4276-935e-6547159741d7 None admin] RBAC: Authorizing `identity:get_user(user_id=abf0cae0793b4db39afb4a7f1e5b6035)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:11.716225 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9190601-cb0a-4276-935e-6547159741d7 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:11.718511 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 110/205] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:11 2022] GET /identity/v3/users/abf0cae0793b4db39afb4a7f1e5b6035 => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:11.723859 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c367f-bd61-4ed9-a6e9-37f42f69c771 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:11.869796 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c367f-bd61-4ed9-a6e9-37f42f69c771 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:11.904604 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:11.918602 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3d109-de72-4ae8-8a78-1d4232e18477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:11.919488 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3d109-de72-4ae8-8a78-1d4232e18477 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:11.919941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3d109-de72-4ae8-8a78-1d4232e18477 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:11.920386 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3d109-de72-4ae8-8a78-1d4232e18477 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:12.008118 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3d109-de72-4ae8-8a78-1d4232e18477 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:12.009917 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3d109-de72-4ae8-8a78-1d4232e18477 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:12.134574 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 96/206] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:12.140658 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9190601-cb0a-4276-935e-6547159741d7 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:12.266366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:12.303940 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9190601-cb0a-4276-935e-6547159741d7 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:12.342611 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa1748-3883-47d2-aef2-56b0b3d79ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:12.343387 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa1748-3883-47d2-aef2-56b0b3d79ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:12.343753 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa1748-3883-47d2-aef2-56b0b3d79ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:12.344120 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa1748-3883-47d2-aef2-56b0b3d79ce3 None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:12.345379 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaa1748-3883-47d2-aef2-56b0b3d79ce3 None admin] RBAC: Authorizing `identity:get_project(project_id=4367ec2ce611409a85975a2a505f1ff8)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:12.347964 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaa1748-3883-47d2-aef2-56b0b3d79ce3 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:12.350424 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 111/207] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:12 2022] GET /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:12.357587 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3d109-de72-4ae8-8a78-1d4232e18477 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:12.393135 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:12.510034 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3d109-de72-4ae8-8a78-1d4232e18477 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:12.564026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f07c2-354b-4368-8613-80e808ee7006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:12.564558 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f07c2-354b-4368-8613-80e808ee7006 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:12.564837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f07c2-354b-4368-8613-80e808ee7006 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:12.565111 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f07c2-354b-4368-8613-80e808ee7006 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:12.565877 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f07c2-354b-4368-8613-80e808ee7006 None admin] RBAC: Adding query filter params (role.id=c4b36aca8aba43d09071f52c763a80a8, scope.project.id=4367ec2ce611409a85975a2a505f1ff8, user.id=abf0cae0793b4db39afb4a7f1e5b6035) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:12.566188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f07c2-354b-4368-8613-80e808ee7006 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:12.567792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f07c2-354b-4368-8613-80e808ee7006 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:12.578916 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 97/208] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 19:59:12 2022] GET /identity/v3/role_assignments?user.id=abf0cae0793b4db39afb4a7f1e5b6035&scope.project.id=4367ec2ce611409a85975a2a505f1ff8&role.id=c4b36aca8aba43d09071f52c763a80a8 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:12.864740 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce57fc8-9bf9-406a-9576-d9a7f8da234d None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:12.865014 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce57fc8-9bf9-406a-9576-d9a7f8da234d None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:12.865530 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce57fc8-9bf9-406a-9576-d9a7f8da234d None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:12.867148 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 112/209] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:12.877481 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4058f58-f6f7-48f2-95a0-1e6e62e41710 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:12.878094 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4058f58-f6f7-48f2-95a0-1e6e62e41710 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:12.878568 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4058f58-f6f7-48f2-95a0-1e6e62e41710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:12.902484 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-c4058f58-f6f7-48f2-95a0-1e6e62e41710 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:12.904157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c4058f58-f6f7-48f2-95a0-1e6e62e41710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:12.988339 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 98/210] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:13.032690 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf728b-d7db-4e29-b1c4-3639138b8eae None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:13.033298 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf728b-d7db-4e29-b1c4-3639138b8eae None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:13.033810 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf728b-d7db-4e29-b1c4-3639138b8eae None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:13.035443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 113/211] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:13.041032 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efb159-ebe7-4ee7-b253-30cedab24d4d None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:13.041340 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efb159-ebe7-4ee7-b253-30cedab24d4d None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:13.041617 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efb159-ebe7-4ee7-b253-30cedab24d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:13.070991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-d4efb159-ebe7-4ee7-b253-30cedab24d4d None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:13.072219 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d4efb159-ebe7-4ee7-b253-30cedab24d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:13.091245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:13.163317 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 99/212] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:13.301030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf728b-d7db-4e29-b1c4-3639138b8eae None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:13.331112 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96c262-52f7-4ba3-b886-16a24c2f2484 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:13.331869 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96c262-52f7-4ba3-b886-16a24c2f2484 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:13.332334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96c262-52f7-4ba3-b886-16a24c2f2484 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:13.333901 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 100/213] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:59:13.339116 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd08fb2-4cc4-40f2-80c5-092ec2a0cca6 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:13.340205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd08fb2-4cc4-40f2-80c5-092ec2a0cca6 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:13.340692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd08fb2-4cc4-40f2-80c5-092ec2a0cca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:13.360300 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-6dd08fb2-4cc4-40f2-80c5-092ec2a0cca6 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:13.361844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-6dd08fb2-4cc4-40f2-80c5-092ec2a0cca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:13.426261 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf728b-d7db-4e29-b1c4-3639138b8eae None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:13.456242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 101/214] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:59:13.465530 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd08fb2-4cc4-40f2-80c5-092ec2a0cca6 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:13.476619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:13.485029 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d3127-353f-4abd-a71e-250979fbadf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:13.485681 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d3127-353f-4abd-a71e-250979fbadf4 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:13.486005 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d3127-353f-4abd-a71e-250979fbadf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:13.486276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d3127-353f-4abd-a71e-250979fbadf4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:13.487730 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d3127-353f-4abd-a71e-250979fbadf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:13.489951 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d3127-353f-4abd-a71e-250979fbadf4 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:13.491548 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 114/215] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:13.498199 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d3127-353f-4abd-a71e-250979fbadf4 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:13.565665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:13.590979 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd08fb2-4cc4-40f2-80c5-092ec2a0cca6 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:13.603123 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d3127-353f-4abd-a71e-250979fbadf4 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:13.631046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbaccd8-8652-47f3-b8b3-3d636db01d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:13.631692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbaccd8-8652-47f3-b8b3-3d636db01d25 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:13.631972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbaccd8-8652-47f3-b8b3-3d636db01d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:13.632236 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbaccd8-8652-47f3-b8b3-3d636db01d25 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:13.643381 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a24319-fecf-4430-9c96-8ab9c9f92a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:13.644187 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a24319-fecf-4430-9c96-8ab9c9f92a5a None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:13.644533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbaccd8-8652-47f3-b8b3-3d636db01d25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:13.644790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a24319-fecf-4430-9c96-8ab9c9f92a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:13.645198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a24319-fecf-4430-9c96-8ab9c9f92a5a None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:13.646036 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a24319-fecf-4430-9c96-8ab9c9f92a5a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:13.647642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a24319-fecf-4430-9c96-8ab9c9f92a5a None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:13.649433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-2dbaccd8-8652-47f3-b8b3-3d636db01d25 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:13.649433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:13.649433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:13.649433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:13.649433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:13.649433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:13.649433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:13.649433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:13.649433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:13.649433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:13.649433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:13.649433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:13.649433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:13.649433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:13.649433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:13.649433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:13.650509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:13.650509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:13.650509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:13.650509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:13.650509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:13.650509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:13.650509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:13.650509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:13.650509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:13.650509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:13.650509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:13.650509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:13.650509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:13.650509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:13.650509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:13.650509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:13.651693 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:13.651693 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:13.651693 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:13.651693 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:13.651693 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:13.651693 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:13.651693 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:13.651693 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:13.651693 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:13.651693 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:13.651693 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:13.651693 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:13.651693 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 102/216] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:13.652763 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2b469-4f9b-44ca-921f-b61c1886d706 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:13.653133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2b469-4f9b-44ca-921f-b61c1886d706 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:13.653467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2b469-4f9b-44ca-921f-b61c1886d706 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:13.654826 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 103/217] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:13.656737 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c2b469-4f9b-44ca-921f-b61c1886d706 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:13.681470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 115/218] 10.222.0.73 () {60 vars in 1132 bytes} [Mon Aug 29 19:59:13 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:59:13.682673 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd95e04-35ba-44fe-b683-b146332c3817 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:13.682958 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd95e04-35ba-44fe-b683-b146332c3817 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:13.683852 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd95e04-35ba-44fe-b683-b146332c3817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:13.704345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-dfd95e04-35ba-44fe-b683-b146332c3817 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:13.705749 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-dfd95e04-35ba-44fe-b683-b146332c3817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:13.775915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:13.792633 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c2b469-4f9b-44ca-921f-b61c1886d706 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:13.809742 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 116/219] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:13.809742 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd95e04-35ba-44fe-b683-b146332c3817 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:13.832744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ca49e-d7cf-435a-bf17-9aba725932f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:13.833485 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ca49e-d7cf-435a-bf17-9aba725932f0 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:13.833846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ca49e-d7cf-435a-bf17-9aba725932f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:13.834226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ca49e-d7cf-435a-bf17-9aba725932f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:13.835027 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ca49e-d7cf-435a-bf17-9aba725932f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:13.835329 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ca49e-d7cf-435a-bf17-9aba725932f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:13.836720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ca49e-d7cf-435a-bf17-9aba725932f0 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:13.851248 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 104/220] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:13.857043 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ca49e-d7cf-435a-bf17-9aba725932f0 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:13.861069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:13.969488 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd95e04-35ba-44fe-b683-b146332c3817 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:13.991862 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ca49e-d7cf-435a-bf17-9aba725932f0 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:14.008015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7f2cb-89f4-4598-9420-a70cca521a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:14.008658 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7f2cb-89f4-4598-9420-a70cca521a1f None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:14.008935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7f2cb-89f4-4598-9420-a70cca521a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:14.009198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7f2cb-89f4-4598-9420-a70cca521a1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:14.021435 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c7f2cb-89f4-4598-9420-a70cca521a1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:14.026452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-a0c7f2cb-89f4-4598-9420-a70cca521a1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:59:14.026452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:14.026452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:14.026452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:14.026452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:14.026452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:14.026452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:14.026452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:14.026452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:14.026452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:14.026452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:14.026452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:14.026452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:14.026452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:14.026452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:14.026452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:14.027849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:14.027849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:14.027849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:14.027849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:14.027849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:14.027849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:14.027849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:14.027849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:14.027849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:14.027849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:14.027849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:14.027849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:14.027849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:14.027849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:14.027849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:14.027849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:14.028645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:14.028645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:14.028645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:14.028645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:14.028645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:14.028645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:14.028645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:14.028645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:14.028645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:14.028645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:14.028645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:59:14.028645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:14.028645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 117/221] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 19:59:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:14.036198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7f2cb-89f4-4598-9420-a70cca521a1f None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:14.049152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35793a3-14d6-4ff5-98b3-d7b3d74a24af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:14.050133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35793a3-14d6-4ff5-98b3-d7b3d74a24af None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:14.050570 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35793a3-14d6-4ff5-98b3-d7b3d74a24af None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:14.051400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35793a3-14d6-4ff5-98b3-d7b3d74a24af None admin] PATH_INFO: `/v3/users/8394d11508ae4a478697ddaf579deccb` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:14.072279 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35793a3-14d6-4ff5-98b3-d7b3d74a24af None admin] RBAC: Authorizing `identity:get_user(user_id=8394d11508ae4a478697ddaf579deccb)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:14.074933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35793a3-14d6-4ff5-98b3-d7b3d74a24af None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:14.077440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 105/222] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:13 2022] GET /identity/v3/users/8394d11508ae4a478697ddaf579deccb => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:14.083530 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35793a3-14d6-4ff5-98b3-d7b3d74a24af None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:14.103666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:14.159533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:14.184507 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7f2cb-89f4-4598-9420-a70cca521a1f None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:14.213355 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35793a3-14d6-4ff5-98b3-d7b3d74a24af None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:14.228452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bee184-7a98-4d1b-a434-211498d0944a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:14.229153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bee184-7a98-4d1b-a434-211498d0944a None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:14.229439 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bee184-7a98-4d1b-a434-211498d0944a None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:14.230666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bee184-7a98-4d1b-a434-211498d0944a None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:14.231908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bee184-7a98-4d1b-a434-211498d0944a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:14.232221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bee184-7a98-4d1b-a434-211498d0944a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:14.234198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bee184-7a98-4d1b-a434-211498d0944a None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:14.244655 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 118/223] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:14.246688 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bee184-7a98-4d1b-a434-211498d0944a None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:14.255326 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07f455-8802-4dc1-abb3-bfd6570b42ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:14.255941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07f455-8802-4dc1-abb3-bfd6570b42ce None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:14.256215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07f455-8802-4dc1-abb3-bfd6570b42ce None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:14.256483 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07f455-8802-4dc1-abb3-bfd6570b42ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:14.338770 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af07f455-8802-4dc1-abb3-bfd6570b42ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:14.341273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af07f455-8802-4dc1-abb3-bfd6570b42ce None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:14.376158 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bee184-7a98-4d1b-a434-211498d0944a None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:14.385889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:14.414281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9932c6-8219-4675-a9c0-af9427e8f85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:14.415216 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9932c6-8219-4675-a9c0-af9427e8f85c None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:14.415660 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9932c6-8219-4675-a9c0-af9427e8f85c None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:14.416115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9932c6-8219-4675-a9c0-af9427e8f85c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:14.423618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:14.433086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9932c6-8219-4675-a9c0-af9427e8f85c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:14.437902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-9b9932c6-8219-4675-a9c0-af9427e8f85c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:14.437902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:14.437902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:14.437902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:14.437902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:14.437902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:14.437902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:14.437902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:14.437902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:14.437902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:14.437902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:14.437902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:14.437902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:14.437902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:14.437902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:14.437902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:14.440195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:14.440195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:14.440195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:14.440195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:14.440195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:14.440195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:14.440195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:14.440195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:14.440195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:14.440195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:14.440195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:14.440195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:14.440195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:14.440195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:14.440195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:14.440195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:14.442313 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:14.442313 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:14.442313 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:14.442313 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:14.442313 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:14.442313 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:14.442313 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:14.442313 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:14.442313 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:14.442313 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:14.442313 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:14.442313 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:14.442313 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 119/224] 10.222.0.73 () {56 vars in 1114 bytes} [Mon Aug 29 19:59:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:14.449989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9932c6-8219-4675-a9c0-af9427e8f85c None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:14.477049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 106/225] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:14.478048 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07f455-8802-4dc1-abb3-bfd6570b42ce None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:14.566178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9932c6-8219-4675-a9c0-af9427e8f85c None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:14.596506 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07f455-8802-4dc1-abb3-bfd6570b42ce None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:14.603386 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0389e2-0a52-4274-b363-39bf61ac050d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:14.604033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0389e2-0a52-4274-b363-39bf61ac050d None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:14.604308 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0389e2-0a52-4274-b363-39bf61ac050d None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:14.604576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0389e2-0a52-4274-b363-39bf61ac050d None admin] PATH_INFO: `/v3/users/217929a84219411991de135622e7c036` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:14.606627 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0389e2-0a52-4274-b363-39bf61ac050d None admin] RBAC: Authorizing `identity:get_user(user_id=217929a84219411991de135622e7c036)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:14.608130 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0389e2-0a52-4274-b363-39bf61ac050d None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:14.609831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 120/226] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:14 2022] GET /identity/v3/users/217929a84219411991de135622e7c036 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:14.610612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0389e2-0a52-4274-b363-39bf61ac050d None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:14.637809 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e92fb0-0ecd-416d-b957-7bfdf2536c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:14.638498 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e92fb0-0ecd-416d-b957-7bfdf2536c85 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:14.638823 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e92fb0-0ecd-416d-b957-7bfdf2536c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:14.639088 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e92fb0-0ecd-416d-b957-7bfdf2536c85 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:14.639757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e92fb0-0ecd-416d-b957-7bfdf2536c85 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:14.640058 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e92fb0-0ecd-416d-b957-7bfdf2536c85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:14.641484 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e92fb0-0ecd-416d-b957-7bfdf2536c85 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:14.650434 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 107/227] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 19:59:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:14.651322 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e92fb0-0ecd-416d-b957-7bfdf2536c85 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:14.662171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:14.679690 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:14.737400 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0389e2-0a52-4274-b363-39bf61ac050d None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:14.758863 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e92fb0-0ecd-416d-b957-7bfdf2536c85 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:14.776073 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2605a2-dd4f-49e8-a752-2256775ed909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:14.776847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2605a2-dd4f-49e8-a752-2256775ed909 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:14.777697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2605a2-dd4f-49e8-a752-2256775ed909 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:14.778123 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2605a2-dd4f-49e8-a752-2256775ed909 None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:14.779393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2605a2-dd4f-49e8-a752-2256775ed909 None admin] RBAC: Authorizing `identity:get_project(project_id=2e1ab10529d34aaabbf5bdd2cff84222)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:14.780847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2605a2-dd4f-49e8-a752-2256775ed909 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:14.782635 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 121/228] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:14 2022] GET /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:14.783410 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2605a2-dd4f-49e8-a752-2256775ed909 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:14.803082 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5062e1-bfd1-40a6-b1c7-a4b78ae8d64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:14.803981 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5062e1-bfd1-40a6-b1c7-a4b78ae8d64f None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:14.803981 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5062e1-bfd1-40a6-b1c7-a4b78ae8d64f None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:14.804252 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5062e1-bfd1-40a6-b1c7-a4b78ae8d64f None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:14.805381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5062e1-bfd1-40a6-b1c7-a4b78ae8d64f None admin] RBAC: Authorizing `identity:get_project(project_id=2e1ab10529d34aaabbf5bdd2cff84222)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:14.806859 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5062e1-bfd1-40a6-b1c7-a4b78ae8d64f None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:14.809573 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 108/229] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:14 2022] GET /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:14.810454 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5062e1-bfd1-40a6-b1c7-a4b78ae8d64f None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:14.906474 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2605a2-dd4f-49e8-a752-2256775ed909 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:14.909106 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:14.917312 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:14.927594 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5062e1-bfd1-40a6-b1c7-a4b78ae8d64f None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:14.957931 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d2ab0-8115-47a9-b13f-22be0dc44fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:14.958954 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d2ab0-8115-47a9-b13f-22be0dc44fed None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:14.959409 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d2ab0-8115-47a9-b13f-22be0dc44fed None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:14.959855 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d2ab0-8115-47a9-b13f-22be0dc44fed None admin] PATH_INFO: `/v3/users/217929a84219411991de135622e7c036` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:14.961545 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d2ab0-8115-47a9-b13f-22be0dc44fed None admin] RBAC: Authorizing `identity:get_user(user_id=217929a84219411991de135622e7c036)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:14.963622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d2ab0-8115-47a9-b13f-22be0dc44fed None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:14.966389 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 122/230] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:14 2022] GET /identity/v3/users/217929a84219411991de135622e7c036 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:14.971317 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f006fd1-19b2-4874-b32e-a79daa533e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:14.971575 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f006fd1-19b2-4874-b32e-a79daa533e5d None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:14.971839 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f006fd1-19b2-4874-b32e-a79daa533e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:14.972110 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f006fd1-19b2-4874-b32e-a79daa533e5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:14.972848 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f006fd1-19b2-4874-b32e-a79daa533e5d None admin] RBAC: Adding query filter params (role.id=9265e1fd617b4c0b8fbf267885394060, scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222, user.id=8394d11508ae4a478697ddaf579deccb) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:14.973154 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f006fd1-19b2-4874-b32e-a79daa533e5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:14.974282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d2ab0-8115-47a9-b13f-22be0dc44fed None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:14.974648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f006fd1-19b2-4874-b32e-a79daa533e5d None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:14.983124 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 109/231] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 19:59:14 2022] GET /identity/v3/role_assignments?user.id=8394d11508ae4a478697ddaf579deccb&scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222&role.id=9265e1fd617b4c0b8fbf267885394060 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:14.983971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f006fd1-19b2-4874-b32e-a79daa533e5d None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:15.111386 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d2ab0-8115-47a9-b13f-22be0dc44fed None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:15.154259 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bc970-cd8f-4341-a347-d32073e5efdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:15.155486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bc970-cd8f-4341-a347-d32073e5efdf None admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:15.156016 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bc970-cd8f-4341-a347-d32073e5efdf None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:15.156570 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bc970-cd8f-4341-a347-d32073e5efdf None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222/users/217929a84219411991de135622e7c036/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:15.159355 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21bc970-cd8f-4341-a347-d32073e5efdf None admin] RBAC: Authorizing `identity:create_grant(project_id=2e1ab10529d34aaabbf5bdd2cff84222, user_id=217929a84219411991de135622e7c036, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:15.161133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21bc970-cd8f-4341-a347-d32073e5efdf None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:15.178633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 123/232] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 19:59:14 2022] PUT /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222/users/217929a84219411991de135622e7c036/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:59:15.224525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f006fd1-19b2-4874-b32e-a79daa533e5d None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:15.265079 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc384d53-fb6d-45dc-8909-bb7f409929f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:15.265760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc384d53-fb6d-45dc-8909-bb7f409929f0 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:15.266041 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc384d53-fb6d-45dc-8909-bb7f409929f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:15.266310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc384d53-fb6d-45dc-8909-bb7f409929f0 None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:15.267692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc384d53-fb6d-45dc-8909-bb7f409929f0 None admin] RBAC: Authorizing `identity:get_project(project_id=2e1ab10529d34aaabbf5bdd2cff84222)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:15.269182 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc384d53-fb6d-45dc-8909-bb7f409929f0 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:15.270897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 110/233] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:14 2022] GET /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222 => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:15.277454 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bc970-cd8f-4341-a347-d32073e5efdf None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:15.332298 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:15.430198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bc970-cd8f-4341-a347-d32073e5efdf None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:15.481354 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286e247-6fbf-4ba8-98df-74642eecb8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:15.482394 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286e247-6fbf-4ba8-98df-74642eecb8ab None admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:15.482872 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286e247-6fbf-4ba8-98df-74642eecb8ab None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:15.483365 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286e247-6fbf-4ba8-98df-74642eecb8ab None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222/users/217929a84219411991de135622e7c036/roles/b9184599d95b4950beca34304cdd2eb8` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:15.487631 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286e247-6fbf-4ba8-98df-74642eecb8ab None admin] RBAC: Authorizing `identity:create_grant(project_id=2e1ab10529d34aaabbf5bdd2cff84222, user_id=217929a84219411991de135622e7c036, role_id=b9184599d95b4950beca34304cdd2eb8)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:15.489113 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286e247-6fbf-4ba8-98df-74642eecb8ab None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:15.505550 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 124/234] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 19:59:15 2022] PUT /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222/users/217929a84219411991de135622e7c036/roles/b9184599d95b4950beca34304cdd2eb8 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:59:16.351647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6707f-05ac-4ac0-9347-aa58cd19b68f None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:16.352145 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6707f-05ac-4ac0-9347-aa58cd19b68f None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:16.352145 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6707f-05ac-4ac0-9347-aa58cd19b68f None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:16.353506 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 111/235] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:16.355502 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5fc91-23ee-41b8-9709-dcf1367feaeb None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:16.355797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5fc91-23ee-41b8-9709-dcf1367feaeb None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:16.356066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5fc91-23ee-41b8-9709-dcf1367feaeb None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:16.357531 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 125/236] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:16.363520 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779898c-4972-4350-afad-0088c0edaf28 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:16.363800 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779898c-4972-4350-afad-0088c0edaf28 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:16.364071 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779898c-4972-4350-afad-0088c0edaf28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:16.366889 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2724f-f67c-4521-a243-57e20a676e78 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:16.367168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2724f-f67c-4521-a243-57e20a676e78 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:16.367437 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2724f-f67c-4521-a243-57e20a676e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:16.397180 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-faa2724f-f67c-4521-a243-57e20a676e78 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:16.398413 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-faa2724f-f67c-4521-a243-57e20a676e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:16.405099 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-3779898c-4972-4350-afad-0088c0edaf28 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:16.406324 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-3779898c-4972-4350-afad-0088c0edaf28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:16.446786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:16.541977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 126/237] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:16.556189 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 112/238] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:17.001451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779898c-4972-4350-afad-0088c0edaf28 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:17.032892 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:17.130148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2724f-f67c-4521-a243-57e20a676e78 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:17.130809 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779898c-4972-4350-afad-0088c0edaf28 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:17.192214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ddab44-d41f-49ab-925c-5eb2a59dc5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:17.192858 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddab44-d41f-49ab-925c-5eb2a59dc5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:17.193137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddab44-d41f-49ab-925c-5eb2a59dc5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:17.193405 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddab44-d41f-49ab-925c-5eb2a59dc5a1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:17.207568 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ddab44-d41f-49ab-925c-5eb2a59dc5a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:17.213504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-20ddab44-d41f-49ab-925c-5eb2a59dc5a1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 19:59:17.213504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:17.213504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:17.213504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:17.213504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:17.213504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:17.213504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:17.213504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:17.213504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:17.213504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:17.213504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:17.213504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:17.213504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:17.213504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:17.213504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:17.213504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:17.214495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:17.214495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:17.214495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:17.214495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:17.214495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:17.214495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:17.214495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:17.214495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:17.214495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:17.214495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:17.214495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:17.214495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:17.214495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:17.214495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:17.214495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:17.214495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:17.215282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:17.215282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:17.215282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:17.215282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:17.215282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:17.215282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:17.215282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:17.215282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:17.215282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:17.215282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:17.215282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 19:59:17.215282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:17.216057 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 127/239] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:17.221727 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ddab44-d41f-49ab-925c-5eb2a59dc5a1 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:17.327136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2724f-f67c-4521-a243-57e20a676e78 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:17.349770 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:17.374916 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ddab44-d41f-49ab-925c-5eb2a59dc5a1 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:17.376517 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53dd60-453a-4fce-84fc-53dc7d44e21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:17.377361 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53dd60-453a-4fce-84fc-53dc7d44e21d None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:17.377658 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53dd60-453a-4fce-84fc-53dc7d44e21d None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:17.377953 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53dd60-453a-4fce-84fc-53dc7d44e21d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:17.379890 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53dd60-453a-4fce-84fc-53dc7d44e21d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:17.381798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53dd60-453a-4fce-84fc-53dc7d44e21d None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:17.383549 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 113/240] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:17.384814 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ae7e1-5b6b-4802-a9a8-ddd83f68a3f9 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:17.385160 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ae7e1-5b6b-4802-a9a8-ddd83f68a3f9 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:17.385436 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ae7e1-5b6b-4802-a9a8-ddd83f68a3f9 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:17.387414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 114/241] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:17.392866 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ae7e1-5b6b-4802-a9a8-ddd83f68a3f9 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:17.423440 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbdd47-0ada-43c3-8f68-366050258894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:17.424550 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbdd47-0ada-43c3-8f68-366050258894 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:17.425007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbdd47-0ada-43c3-8f68-366050258894 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:17.426488 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbdd47-0ada-43c3-8f68-366050258894 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:17.428249 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbdd47-0ada-43c3-8f68-366050258894 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:17.429006 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbdd47-0ada-43c3-8f68-366050258894 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:17.431756 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbdd47-0ada-43c3-8f68-366050258894 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:17.434847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:17.446023 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 128/242] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:17.448402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0656b-b627-442e-8908-dcefe52ccd0d None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:17.449739 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0656b-b627-442e-8908-dcefe52ccd0d None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:17.450275 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0656b-b627-442e-8908-dcefe52ccd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:17.477662 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-92e0656b-b627-442e-8908-dcefe52ccd0d None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:17.480897 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-92e0656b-b627-442e-8908-dcefe52ccd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:17.536329 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ae7e1-5b6b-4802-a9a8-ddd83f68a3f9 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:17.585595 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 129/243] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:17.587515 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5b3ad-4a7a-487e-a66e-9f5be87f80b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:17.587645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0656b-b627-442e-8908-dcefe52ccd0d None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:17.588147 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5b3ad-4a7a-487e-a66e-9f5be87f80b1 None admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:17.588446 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5b3ad-4a7a-487e-a66e-9f5be87f80b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:17.588694 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5b3ad-4a7a-487e-a66e-9f5be87f80b1 None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:17.595745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5b3ad-4a7a-487e-a66e-9f5be87f80b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:17.597326 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5b3ad-4a7a-487e-a66e-9f5be87f80b1 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:17.639653 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 115/244] 10.222.0.73 () {60 vars in 1132 bytes} [Mon Aug 29 19:59:17 2022] POST /identity/v3/users => generated 293 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:59:17.648255 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ea166-4dea-4552-bb6a-1b7a14f96fee None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:17.648593 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ea166-4dea-4552-bb6a-1b7a14f96fee None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:17.648842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ea166-4dea-4552-bb6a-1b7a14f96fee None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:17.650281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 116/245] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:17.659963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f66e3-52ec-41d4-9e65-7e3de38c82cf None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:17.660346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f66e3-52ec-41d4-9e65-7e3de38c82cf None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:17.660515 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f66e3-52ec-41d4-9e65-7e3de38c82cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:17.704710 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:17.708759 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-895f66e3-52ec-41d4-9e65-7e3de38c82cf None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:17.710062 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-895f66e3-52ec-41d4-9e65-7e3de38c82cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:17.746830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0656b-b627-442e-8908-dcefe52ccd0d None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:17.797452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e787e1-04cc-44df-869f-e3de787911c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:17.798197 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e787e1-04cc-44df-869f-e3de787911c8 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:17.799886 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e787e1-04cc-44df-869f-e3de787911c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:17.799886 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e787e1-04cc-44df-869f-e3de787911c8 None admin] PATH_INFO: `/v3/users/7f4bbf36f80d4b92aaaed102c0c6d52a` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:17.818697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e787e1-04cc-44df-869f-e3de787911c8 None admin] RBAC: Authorizing `identity:get_user(user_id=7f4bbf36f80d4b92aaaed102c0c6d52a)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:17.820397 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e787e1-04cc-44df-869f-e3de787911c8 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:17.823212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 130/246] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:17 2022] GET /identity/v3/users/7f4bbf36f80d4b92aaaed102c0c6d52a => generated 297 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:17.826543 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8fa2b-ffe1-4583-ad12-9f5d6441132e None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:17.827088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8fa2b-ffe1-4583-ad12-9f5d6441132e None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:17.827578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8fa2b-ffe1-4583-ad12-9f5d6441132e None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:17.829403 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 131/247] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:17.830551 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8fa2b-ffe1-4583-ad12-9f5d6441132e None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:17.836856 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 117/248] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:17.839185 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31dd08-9d4b-4cf2-81fe-8503f32fdd2e None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:17.840721 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31dd08-9d4b-4cf2-81fe-8503f32fdd2e None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:17.841264 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31dd08-9d4b-4cf2-81fe-8503f32fdd2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:17.867282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-9b31dd08-9d4b-4cf2-81fe-8503f32fdd2e None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:17.871755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9b31dd08-9d4b-4cf2-81fe-8503f32fdd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:17.874153 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:17.981292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 118/249] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:17.981292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31dd08-9d4b-4cf2-81fe-8503f32fdd2e None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:17.981578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8fa2b-ffe1-4583-ad12-9f5d6441132e None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:18.034772 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:18.042157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25d365-1c36-441b-8838-58ca29152943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:18.043600 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25d365-1c36-441b-8838-58ca29152943 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:18.044044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25d365-1c36-441b-8838-58ca29152943 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:18.044326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25d365-1c36-441b-8838-58ca29152943 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:18.138666 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31dd08-9d4b-4cf2-81fe-8503f32fdd2e None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:18.176294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25d365-1c36-441b-8838-58ca29152943 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:18.177820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25d365-1c36-441b-8838-58ca29152943 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:18.201085 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d11730-3bd5-49d5-9e2c-0b58cba6a3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:18.202171 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d11730-3bd5-49d5-9e2c-0b58cba6a3e5 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:18.203187 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d11730-3bd5-49d5-9e2c-0b58cba6a3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:18.203677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d11730-3bd5-49d5-9e2c-0b58cba6a3e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:18.213969 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:18.222645 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d11730-3bd5-49d5-9e2c-0b58cba6a3e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:18.229380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-20d11730-3bd5-49d5-9e2c-0b58cba6a3e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:18.229380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:18.229380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:18.229380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:18.229380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:18.229380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:18.229380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:18.229380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:18.229380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:18.229380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:18.229380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:18.229380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:18.229380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:18.229380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:18.229380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:18.229380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:18.230897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:18.230897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:18.230897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:18.230897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:18.230897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:18.230897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:18.230897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:18.230897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:18.230897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:18.230897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:18.230897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:18.230897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:18.230897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:18.230897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:18.230897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:18.230897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:18.231678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:18.231678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:18.231678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:18.231678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:18.231678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:18.231678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:18.231678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:18.231678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:18.231678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:18.231678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:18.231678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:18.231678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:18.234094 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 119/250] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:18.235094 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d11730-3bd5-49d5-9e2c-0b58cba6a3e5 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:18.324777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 132/251] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:18.326490 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25d365-1c36-441b-8838-58ca29152943 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:18.375000 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:18.376824 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d11730-3bd5-49d5-9e2c-0b58cba6a3e5 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:18.421482 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5643e2-a295-4155-b255-7ae890ddde53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:18.422581 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5643e2-a295-4155-b255-7ae890ddde53 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:18.423003 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5643e2-a295-4155-b255-7ae890ddde53 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:18.423440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5643e2-a295-4155-b255-7ae890ddde53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:18.440930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5643e2-a295-4155-b255-7ae890ddde53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:18.451269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-8a5643e2-a295-4155-b255-7ae890ddde53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:59:18.451269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:18.451269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:18.451269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:18.451269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:18.451269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:18.451269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:18.451269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:18.451269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:18.451269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:18.451269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:18.451269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:18.451269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:18.451269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:18.451269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:18.451269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:18.452211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:18.452211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:18.452211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:18.452211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:18.452211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:18.452211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:18.452211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:18.452211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:18.452211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:18.452211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:18.452211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:18.452211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:18.452211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:18.452211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:18.452211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:18.452211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:18.453067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:18.453067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:18.453067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:18.453067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:18.453067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:18.453067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:18.453067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:18.453067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:18.453067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:18.453067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:18.453067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:59:18.453067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:18.453670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 120/252] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 19:59:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:18.457732 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5643e2-a295-4155-b255-7ae890ddde53 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:18.470012 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25d365-1c36-441b-8838-58ca29152943 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:18.519721 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1c169-6478-41bd-b930-1fc41aad3f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:18.520347 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1c169-6478-41bd-b930-1fc41aad3f85 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:18.520649 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1c169-6478-41bd-b930-1fc41aad3f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:18.521166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1c169-6478-41bd-b930-1fc41aad3f85 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:18.521878 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1c169-6478-41bd-b930-1fc41aad3f85 None admin] RBAC: Adding query filter params (name=member) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:18.522219 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1c169-6478-41bd-b930-1fc41aad3f85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:18.524010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1c169-6478-41bd-b930-1fc41aad3f85 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:18.536104 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:18.536454 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 133/253] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:18.539230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1c169-6478-41bd-b930-1fc41aad3f85 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:18.595047 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5643e2-a295-4155-b255-7ae890ddde53 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:18.646918 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af141b9e-1bd9-4735-a207-8cd545c13829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:18.647489 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af141b9e-1bd9-4735-a207-8cd545c13829 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:18.647750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af141b9e-1bd9-4735-a207-8cd545c13829 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:18.648022 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af141b9e-1bd9-4735-a207-8cd545c13829 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:18.660175 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af141b9e-1bd9-4735-a207-8cd545c13829 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:18.665105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-af141b9e-1bd9-4735-a207-8cd545c13829 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:18.665105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:18.665105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:18.665105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:18.665105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:18.665105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:18.665105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:18.665105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:18.665105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:18.665105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:18.665105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:18.665105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:18.665105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:18.665105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:18.665105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:18.665105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:18.666214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:18.666214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:18.666214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:18.666214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:18.666214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:18.666214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:18.666214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:18.666214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:18.666214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:18.666214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:18.666214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:18.666214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:18.666214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:18.666214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:18.666214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:18.666214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:18.666986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:18.666986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:18.666986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:18.666986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:18.666986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:18.666986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:18.666986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:18.666986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:18.666986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:18.666986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:18.666986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:18.666986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:18.666986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 121/254] 10.222.0.73 () {56 vars in 1114 bytes} [Mon Aug 29 19:59:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:18.672235 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af141b9e-1bd9-4735-a207-8cd545c13829 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:18.673007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:18.681521 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1c169-6478-41bd-b930-1fc41aad3f85 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:18.720785 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac6a5d-f29f-42b9-8739-f128ba3ce978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:18.721897 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac6a5d-f29f-42b9-8739-f128ba3ce978 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:18.722373 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac6a5d-f29f-42b9-8739-f128ba3ce978 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:18.723308 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac6a5d-f29f-42b9-8739-f128ba3ce978 None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:18.724857 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac6a5d-f29f-42b9-8739-f128ba3ce978 None admin] RBAC: Authorizing `identity:get_project(project_id=2e1ab10529d34aaabbf5bdd2cff84222)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:18.726711 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac6a5d-f29f-42b9-8739-f128ba3ce978 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:18.728641 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 134/255] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:18 2022] GET /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:18.729644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac6a5d-f29f-42b9-8739-f128ba3ce978 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:18.787453 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af141b9e-1bd9-4735-a207-8cd545c13829 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:18.805600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:18.825467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8f717-5df4-47a7-a691-a8fa252ff845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:18.826066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8f717-5df4-47a7-a691-a8fa252ff845 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:18.826313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8f717-5df4-47a7-a691-a8fa252ff845 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:18.826643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8f717-5df4-47a7-a691-a8fa252ff845 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:18.827326 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8f717-5df4-47a7-a691-a8fa252ff845 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:18.827630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8f717-5df4-47a7-a691-a8fa252ff845 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:18.829119 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8f717-5df4-47a7-a691-a8fa252ff845 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:18.843332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 122/256] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:18.843332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8f717-5df4-47a7-a691-a8fa252ff845 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:18.856199 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac6a5d-f29f-42b9-8739-f128ba3ce978 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:18.897340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41faf7-5351-456f-ba27-6e9fc5b8c3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:18.898431 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41faf7-5351-456f-ba27-6e9fc5b8c3c6 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:18.898926 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41faf7-5351-456f-ba27-6e9fc5b8c3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:18.899389 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41faf7-5351-456f-ba27-6e9fc5b8c3c6 None admin] PATH_INFO: `/v3/users/8394d11508ae4a478697ddaf579deccb` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:18.900946 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c41faf7-5351-456f-ba27-6e9fc5b8c3c6 None admin] RBAC: Authorizing `identity:get_user(user_id=8394d11508ae4a478697ddaf579deccb)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:18.902696 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c41faf7-5351-456f-ba27-6e9fc5b8c3c6 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:18.905152 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 135/257] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:18 2022] GET /identity/v3/users/8394d11508ae4a478697ddaf579deccb => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:18.906349 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41faf7-5351-456f-ba27-6e9fc5b8c3c6 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:18.940532 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:18.952290 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8f717-5df4-47a7-a691-a8fa252ff845 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:18.989886 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e033e78-074c-40f6-a888-672c05225a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:18.990512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e033e78-074c-40f6-a888-672c05225a8b None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:18.990793 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e033e78-074c-40f6-a888-672c05225a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:18.991071 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e033e78-074c-40f6-a888-672c05225a8b None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:18.991743 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e033e78-074c-40f6-a888-672c05225a8b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:18.992048 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e033e78-074c-40f6-a888-672c05225a8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:18.993700 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e033e78-074c-40f6-a888-672c05225a8b None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:19.003481 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 123/258] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 19:59:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:19.009074 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e033e78-074c-40f6-a888-672c05225a8b None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:19.022761 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41faf7-5351-456f-ba27-6e9fc5b8c3c6 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:19.054485 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:19.063099 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c949095-40ed-4357-b5b4-71651752e3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:19.064033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c949095-40ed-4357-b5b4-71651752e3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:19.064546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c949095-40ed-4357-b5b4-71651752e3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:19.065037 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c949095-40ed-4357-b5b4-71651752e3c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:19.066056 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c949095-40ed-4357-b5b4-71651752e3c7 None admin] RBAC: Adding query filter params (role.id=c4b36aca8aba43d09071f52c763a80a8, scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222, user.id=7f4bbf36f80d4b92aaaed102c0c6d52a) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:19.066569 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c949095-40ed-4357-b5b4-71651752e3c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:19.068425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c949095-40ed-4357-b5b4-71651752e3c7 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:19.075757 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 136/259] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 19:59:18 2022] GET /identity/v3/role_assignments?user.id=7f4bbf36f80d4b92aaaed102c0c6d52a&scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222&role.id=c4b36aca8aba43d09071f52c763a80a8 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:19.076851 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c949095-40ed-4357-b5b4-71651752e3c7 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:19.155883 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e033e78-074c-40f6-a888-672c05225a8b None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:19.193079 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b620c769-9dc2-4361-844b-d0d2cb6f180f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:19.193736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620c769-9dc2-4361-844b-d0d2cb6f180f None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:19.194035 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620c769-9dc2-4361-844b-d0d2cb6f180f None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:19.194316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620c769-9dc2-4361-844b-d0d2cb6f180f None admin] PATH_INFO: `/v3/users/217929a84219411991de135622e7c036` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:19.195482 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b620c769-9dc2-4361-844b-d0d2cb6f180f None admin] RBAC: Authorizing `identity:get_user(user_id=217929a84219411991de135622e7c036)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:19.202632 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b620c769-9dc2-4361-844b-d0d2cb6f180f None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:19.202632 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 124/260] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:19 2022] GET /identity/v3/users/217929a84219411991de135622e7c036 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:19.202632 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b620c769-9dc2-4361-844b-d0d2cb6f180f None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:19.219383 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:19.228495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c949095-40ed-4357-b5b4-71651752e3c7 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:19.338457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea677da-7e4b-43a1-9759-d5309d4ce5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:19.339115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea677da-7e4b-43a1-9759-d5309d4ce5ed None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:19.339399 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea677da-7e4b-43a1-9759-d5309d4ce5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:19.339729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea677da-7e4b-43a1-9759-d5309d4ce5ed None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:19.341087 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea677da-7e4b-43a1-9759-d5309d4ce5ed None admin] RBAC: Authorizing `identity:get_project(project_id=2e1ab10529d34aaabbf5bdd2cff84222)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:19.342583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea677da-7e4b-43a1-9759-d5309d4ce5ed None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:19.344273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 137/261] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:19 2022] GET /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222 => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:19.345056 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea677da-7e4b-43a1-9759-d5309d4ce5ed None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:19.359549 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b620c769-9dc2-4361-844b-d0d2cb6f180f None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:19.383632 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:19.401493 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad892487-dedc-488d-a16a-efcc46d7f8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:19.402228 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad892487-dedc-488d-a16a-efcc46d7f8e4 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:19.402522 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad892487-dedc-488d-a16a-efcc46d7f8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:19.402875 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad892487-dedc-488d-a16a-efcc46d7f8e4 None admin] PATH_INFO: `/v3/users/217929a84219411991de135622e7c036` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:19.404433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad892487-dedc-488d-a16a-efcc46d7f8e4 None admin] RBAC: Authorizing `identity:get_user(user_id=217929a84219411991de135622e7c036)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:19.406271 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad892487-dedc-488d-a16a-efcc46d7f8e4 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:19.408283 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 125/262] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:19 2022] GET /identity/v3/users/217929a84219411991de135622e7c036 => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:19.414439 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad892487-dedc-488d-a16a-efcc46d7f8e4 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:19.527969 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad892487-dedc-488d-a16a-efcc46d7f8e4 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:19.570259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d3234-0b93-4b63-86af-384977542b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:19.571184 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d3234-0b93-4b63-86af-384977542b8b None admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:19.571474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d3234-0b93-4b63-86af-384977542b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:19.572224 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d3234-0b93-4b63-86af-384977542b8b None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222/users/8394d11508ae4a478697ddaf579deccb/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:19.581728 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d3234-0b93-4b63-86af-384977542b8b None admin] RBAC: Authorizing `identity:create_grant(project_id=2e1ab10529d34aaabbf5bdd2cff84222, user_id=8394d11508ae4a478697ddaf579deccb, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:19.581728 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d3234-0b93-4b63-86af-384977542b8b None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:19.601204 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 126/263] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 19:59:19 2022] PUT /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222/users/8394d11508ae4a478697ddaf579deccb/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:59:19.602456 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d3234-0b93-4b63-86af-384977542b8b None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:19.605336 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea677da-7e4b-43a1-9759-d5309d4ce5ed None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:19.681609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:19.682732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316db914-dc3d-4767-8bdd-ed25b7b7583b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:19.683491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316db914-dc3d-4767-8bdd-ed25b7b7583b None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:19.683925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316db914-dc3d-4767-8bdd-ed25b7b7583b None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:19.684414 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316db914-dc3d-4767-8bdd-ed25b7b7583b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:19.749888 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:19.794258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d3234-0b93-4b63-86af-384977542b8b None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:19.829205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316db914-dc3d-4767-8bdd-ed25b7b7583b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:19.832315 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316db914-dc3d-4767-8bdd-ed25b7b7583b None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:19.862590 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f4b42-b28c-44da-9786-cbc658207ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:19.863583 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f4b42-b28c-44da-9786-cbc658207ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:19.864037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f4b42-b28c-44da-9786-cbc658207ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:19.864504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f4b42-b28c-44da-9786-cbc658207ce8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:20.004399 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f4b42-b28c-44da-9786-cbc658207ce8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:20.006078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f4b42-b28c-44da-9786-cbc658207ce8 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:20.016401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 138/264] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 672 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:20.022082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316db914-dc3d-4767-8bdd-ed25b7b7583b None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:20.053455 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:20.138080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:20.140729 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 127/265] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:20.146584 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f4b42-b28c-44da-9786-cbc658207ce8 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:20.193284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316db914-dc3d-4767-8bdd-ed25b7b7583b None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:20.261167 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf3803-2332-4a47-a476-65d927ba73f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:20.262502 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf3803-2332-4a47-a476-65d927ba73f9 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:20.262882 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf3803-2332-4a47-a476-65d927ba73f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:20.263360 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf3803-2332-4a47-a476-65d927ba73f9 None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:20.268781 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf3803-2332-4a47-a476-65d927ba73f9 None admin] RBAC: Authorizing `identity:get_project(project_id=2e1ab10529d34aaabbf5bdd2cff84222)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:20.270425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf3803-2332-4a47-a476-65d927ba73f9 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:20.272583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 139/266] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:20 2022] GET /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:20.278269 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf3803-2332-4a47-a476-65d927ba73f9 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:20.312189 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f4b42-b28c-44da-9786-cbc658207ce8 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:20.381294 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a6ce4-9ae6-44bc-83e3-ea15100d9043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:20.381983 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a6ce4-9ae6-44bc-83e3-ea15100d9043 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:20.382271 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a6ce4-9ae6-44bc-83e3-ea15100d9043 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:20.382546 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a6ce4-9ae6-44bc-83e3-ea15100d9043 None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:20.386466 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a6ce4-9ae6-44bc-83e3-ea15100d9043 None admin] RBAC: Authorizing `identity:get_project(project_id=2e1ab10529d34aaabbf5bdd2cff84222)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:20.388463 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a6ce4-9ae6-44bc-83e3-ea15100d9043 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:20.390532 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 128/267] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:20 2022] GET /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:20.391716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fefd8e-ece6-4bb7-b170-2cbfff6d8a35 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:20.391999 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fefd8e-ece6-4bb7-b170-2cbfff6d8a35 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:20.392333 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fefd8e-ece6-4bb7-b170-2cbfff6d8a35 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:20.393688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 129/268] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:20.399445 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fefd8e-ece6-4bb7-b170-2cbfff6d8a35 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:20.417925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:20.438710 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf3803-2332-4a47-a476-65d927ba73f9 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:20.475112 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c3b24-263a-4121-bc7d-83995505dffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:20.475731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c3b24-263a-4121-bc7d-83995505dffa None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:20.476119 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c3b24-263a-4121-bc7d-83995505dffa None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:20.476399 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c3b24-263a-4121-bc7d-83995505dffa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:20.477122 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c3b24-263a-4121-bc7d-83995505dffa None admin] RBAC: Adding query filter params (role.id=296ee2c194824485af71618619f82e09, scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222, user.id=217929a84219411991de135622e7c036) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:20.477430 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c3b24-263a-4121-bc7d-83995505dffa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:20.478930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c3b24-263a-4121-bc7d-83995505dffa None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:20.485791 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 140/269] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 19:59:20 2022] GET /identity/v3/role_assignments?user.id=217929a84219411991de135622e7c036&scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222&role.id=296ee2c194824485af71618619f82e09 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:20.486448 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:20.496260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a77542-66b4-4432-8e83-2daee77ea042 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:20.496729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a77542-66b4-4432-8e83-2daee77ea042 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:20.497175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a77542-66b4-4432-8e83-2daee77ea042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:20.523387 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-d6a77542-66b4-4432-8e83-2daee77ea042 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:20.524535 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-d6a77542-66b4-4432-8e83-2daee77ea042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:20.619271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 141/270] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:20.621460 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54984fd2-adf9-41b0-a6aa-de40d0368e17 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:20.621759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54984fd2-adf9-41b0-a6aa-de40d0368e17 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:20.622035 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54984fd2-adf9-41b0-a6aa-de40d0368e17 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:20.623381 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 142/271] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:20.630908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f521980-8e7b-455f-9f23-4d107744c047 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:20.631184 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f521980-8e7b-455f-9f23-4d107744c047 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:20.631467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f521980-8e7b-455f-9f23-4d107744c047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:20.665878 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-9f521980-8e7b-455f-9f23-4d107744c047 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:20.667116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-9f521980-8e7b-455f-9f23-4d107744c047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:20.674760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fefd8e-ece6-4bb7-b170-2cbfff6d8a35 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:20.742699 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778be1e-ed11-486c-a3e0-20b8de72898d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:20.744771 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778be1e-ed11-486c-a3e0-20b8de72898d None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:20.745057 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778be1e-ed11-486c-a3e0-20b8de72898d None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:20.745593 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778be1e-ed11-486c-a3e0-20b8de72898d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:20.748795 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2778be1e-ed11-486c-a3e0-20b8de72898d None admin] RBAC: Adding query filter params (role.id=b9184599d95b4950beca34304cdd2eb8, scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222, user.id=217929a84219411991de135622e7c036) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:20.749112 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2778be1e-ed11-486c-a3e0-20b8de72898d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:20.752169 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2778be1e-ed11-486c-a3e0-20b8de72898d None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:20.769325 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 130/272] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 19:59:20 2022] GET /identity/v3/role_assignments?user.id=217929a84219411991de135622e7c036&scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222&role.id=b9184599d95b4950beca34304cdd2eb8 => generated 608 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:20.783491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 143/273] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:21.051830 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778be1e-ed11-486c-a3e0-20b8de72898d None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:21.147478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:21.160936 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f521980-8e7b-455f-9f23-4d107744c047 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:21.190572 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778be1e-ed11-486c-a3e0-20b8de72898d None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:21.231932 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:21.237096 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65be75-4856-4116-a5c6-316dc4644bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:21.237762 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65be75-4856-4116-a5c6-316dc4644bcf None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:21.238042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65be75-4856-4116-a5c6-316dc4644bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:21.238310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65be75-4856-4116-a5c6-316dc4644bcf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:21.250781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c65be75-4856-4116-a5c6-316dc4644bcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:21.256467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-1c65be75-4856-4116-a5c6-316dc4644bcf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:21.256467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:21.256467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:21.256467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:21.256467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:21.256467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:21.256467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:21.256467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:21.256467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:21.256467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:21.256467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:21.256467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:21.256467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:21.256467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:21.256467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:21.256467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:21.257348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:21.257348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:21.257348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:21.257348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:21.257348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:21.257348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:21.257348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:21.257348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:21.257348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:21.257348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:21.257348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:21.257348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:21.257348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:21.257348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:21.257348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:21.257348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:21.258223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:21.258223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:21.258223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:21.258223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:21.258223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:21.258223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:21.258223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:21.258223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:21.258223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:21.258223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:21.258223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:21.258223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:21.258223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 131/274] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:21.265182 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65be75-4856-4116-a5c6-316dc4644bcf None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:21.288997 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f521980-8e7b-455f-9f23-4d107744c047 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:21.363579 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4b9e3-c91c-48b0-917f-8dad7a84a909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:21.364937 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4b9e3-c91c-48b0-917f-8dad7a84a909 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:21.369751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4b9e3-c91c-48b0-917f-8dad7a84a909 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:21.369751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4b9e3-c91c-48b0-917f-8dad7a84a909 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:21.369751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4b9e3-c91c-48b0-917f-8dad7a84a909 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:21.371420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4b9e3-c91c-48b0-917f-8dad7a84a909 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:21.373582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 144/275] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:21.382702 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4b9e3-c91c-48b0-917f-8dad7a84a909 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:21.476406 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65be75-4856-4116-a5c6-316dc4644bcf None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:21.541738 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:21.551672 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674ce79-5565-481e-8050-a264c1063ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:21.553794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674ce79-5565-481e-8050-a264c1063ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:21.553794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674ce79-5565-481e-8050-a264c1063ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:21.553794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674ce79-5565-481e-8050-a264c1063ab4 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:21.553794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674ce79-5565-481e-8050-a264c1063ab4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:21.553794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674ce79-5565-481e-8050-a264c1063ab4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:21.555189 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674ce79-5565-481e-8050-a264c1063ab4 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:21.581666 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 132/276] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:21.587773 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674ce79-5565-481e-8050-a264c1063ab4 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:21.588905 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4b9e3-c91c-48b0-917f-8dad7a84a909 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:21.623304 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:21.639375 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3d7a0-2913-4a14-859b-38f6be5e620e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:21.641318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3d7a0-2913-4a14-859b-38f6be5e620e None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:21.642014 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3d7a0-2913-4a14-859b-38f6be5e620e None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:21.642586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3d7a0-2913-4a14-859b-38f6be5e620e None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:21.643947 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb3d7a0-2913-4a14-859b-38f6be5e620e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:21.647012 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb3d7a0-2913-4a14-859b-38f6be5e620e None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:21.680814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 145/277] 10.222.0.73 () {60 vars in 1132 bytes} [Mon Aug 29 19:59:21 2022] POST /identity/v3/users => generated 293 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:59:21.733733 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45479b3-6088-4f6f-a109-1a5860ef5a0b None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:21.733733 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45479b3-6088-4f6f-a109-1a5860ef5a0b None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:21.733733 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45479b3-6088-4f6f-a109-1a5860ef5a0b None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:21.735339 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 146/278] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:59:21.743530 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c780b2-7516-4383-a91a-edf2c6ca5d5a None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:21.744224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c780b2-7516-4383-a91a-edf2c6ca5d5a None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:21.744809 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c780b2-7516-4383-a91a-edf2c6ca5d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:21.769047 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-00c780b2-7516-4383-a91a-edf2c6ca5d5a None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:21.771041 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-00c780b2-7516-4383-a91a-edf2c6ca5d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:21.803538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674ce79-5565-481e-8050-a264c1063ab4 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:21.877600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508a327-23d1-4fb8-a0df-028ff711f2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:21.877600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508a327-23d1-4fb8-a0df-028ff711f2cf None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:21.877600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508a327-23d1-4fb8-a0df-028ff711f2cf None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:21.877600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508a327-23d1-4fb8-a0df-028ff711f2cf None admin] PATH_INFO: `/v3/users/37beb7365bf449d78a8f7ff0c5a996f0` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:21.885641 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 147/279] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:59:21.896964 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508a327-23d1-4fb8-a0df-028ff711f2cf None admin] RBAC: Authorizing `identity:get_user(user_id=37beb7365bf449d78a8f7ff0c5a996f0)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:21.899617 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508a327-23d1-4fb8-a0df-028ff711f2cf None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:21.900301 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 133/280] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:21 2022] GET /identity/v3/users/37beb7365bf449d78a8f7ff0c5a996f0 => generated 293 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:21.910949 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c780b2-7516-4383-a91a-edf2c6ca5d5a None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:22.021782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c780b2-7516-4383-a91a-edf2c6ca5d5a None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:22.033315 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:22.061836 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9661e0a-86c0-4592-b516-2d5dd86dd4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:22.062452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9661e0a-86c0-4592-b516-2d5dd86dd4fd None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:22.062730 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9661e0a-86c0-4592-b516-2d5dd86dd4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:22.062998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9661e0a-86c0-4592-b516-2d5dd86dd4fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:22.063721 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9661e0a-86c0-4592-b516-2d5dd86dd4fd None admin] RBAC: Adding query filter params (role.id=9265e1fd617b4c0b8fbf267885394060, user.id=37beb7365bf449d78a8f7ff0c5a996f0, scope.system=all) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:22.064022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9661e0a-86c0-4592-b516-2d5dd86dd4fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:22.065524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9661e0a-86c0-4592-b516-2d5dd86dd4fd None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:22.071075 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 148/281] 10.222.0.73 () {56 vars in 1308 bytes} [Mon Aug 29 19:59:21 2022] GET /identity/v3/role_assignments?user.id=37beb7365bf449d78a8f7ff0c5a996f0&scope.system=all&role.id=9265e1fd617b4c0b8fbf267885394060 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:22.183086 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508a327-23d1-4fb8-a0df-028ff711f2cf None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:22.191602 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b36600-c9a3-4211-9a19-9c7d92bd85cb None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:22.191894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b36600-c9a3-4211-9a19-9c7d92bd85cb None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:22.192144 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b36600-c9a3-4211-9a19-9c7d92bd85cb None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:22.193863 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 149/282] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:22.199897 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066eccc-3b8f-4540-a511-948ff71aa771 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:22.200205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066eccc-3b8f-4540-a511-948ff71aa771 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:22.200454 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066eccc-3b8f-4540-a511-948ff71aa771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:22.222031 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-a066eccc-3b8f-4540-a511-948ff71aa771 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:22.223247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a066eccc-3b8f-4540-a511-948ff71aa771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:22.256257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:22.330504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 150/283] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:59:22.338566 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508a327-23d1-4fb8-a0df-028ff711f2cf None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:22.389632 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b4a06-87f8-404c-8e3a-505e748f70b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:22.390767 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b4a06-87f8-404c-8e3a-505e748f70b4 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:22.391267 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b4a06-87f8-404c-8e3a-505e748f70b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:22.391739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b4a06-87f8-404c-8e3a-505e748f70b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:22.414647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b4a06-87f8-404c-8e3a-505e748f70b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:22.420698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-6d5b4a06-87f8-404c-8e3a-505e748f70b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 19:59:22.420698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:22.420698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:22.420698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:22.420698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:22.420698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:22.420698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:22.420698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:22.420698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:22.420698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:22.420698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:22.420698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:22.420698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:22.420698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:22.420698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:22.420698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:22.421609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:22.421609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:22.421609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:22.421609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:22.421609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:22.421609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:22.421609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:22.421609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:22.421609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:22.421609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:22.421609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:22.421609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:22.421609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:22.421609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:22.421609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:22.421609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:22.422424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:22.422424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:22.422424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:22.422424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:22.422424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:22.422424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:22.422424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:22.422424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:22.422424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:22.422424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:22.422424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 19:59:22.422424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:22.425056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 134/284] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:22.431054 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066eccc-3b8f-4540-a511-948ff71aa771 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:22.549159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066eccc-3b8f-4540-a511-948ff71aa771 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:22.550673 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:22.599424 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dcaecf-36ca-4df6-8bca-0fc934c184f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:22.600031 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcaecf-36ca-4df6-8bca-0fc934c184f9 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:22.600305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcaecf-36ca-4df6-8bca-0fc934c184f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:22.600574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcaecf-36ca-4df6-8bca-0fc934c184f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:22.601242 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dcaecf-36ca-4df6-8bca-0fc934c184f9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:22.601541 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dcaecf-36ca-4df6-8bca-0fc934c184f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:22.602929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dcaecf-36ca-4df6-8bca-0fc934c184f9 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:22.616661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 151/285] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:22.622963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b4a06-87f8-404c-8e3a-505e748f70b4 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:22.772682 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b4a06-87f8-404c-8e3a-505e748f70b4 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:22.785311 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:22.802882 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dcaecf-36ca-4df6-8bca-0fc934c184f9 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:22.819613 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe9707-f847-4d03-a391-007768727d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:22.820261 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe9707-f847-4d03-a391-007768727d4e None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:22.820540 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe9707-f847-4d03-a391-007768727d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:22.820828 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe9707-f847-4d03-a391-007768727d4e None admin] PATH_INFO: `/v3/users/7f4bbf36f80d4b92aaaed102c0c6d52a` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:22.822003 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe9707-f847-4d03-a391-007768727d4e None admin] RBAC: Authorizing `identity:get_user(user_id=7f4bbf36f80d4b92aaaed102c0c6d52a)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:22.824012 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe9707-f847-4d03-a391-007768727d4e None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:22.826417 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 135/286] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:22 2022] GET /identity/v3/users/7f4bbf36f80d4b92aaaed102c0c6d52a => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:22.832022 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe9707-f847-4d03-a391-007768727d4e None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:22.946747 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe9707-f847-4d03-a391-007768727d4e None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:22.990783 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a1190-2084-4b75-86b3-9bff1bb92d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:22.991417 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a1190-2084-4b75-86b3-9bff1bb92d81 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:22.991694 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a1190-2084-4b75-86b3-9bff1bb92d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:22.991964 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a1190-2084-4b75-86b3-9bff1bb92d81 None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:23.001282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a1190-2084-4b75-86b3-9bff1bb92d81 None admin] RBAC: Authorizing `identity:get_project(project_id=2e1ab10529d34aaabbf5bdd2cff84222)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:23.002757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a1190-2084-4b75-86b3-9bff1bb92d81 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:23.004430 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 136/287] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:22 2022] GET /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:23.010108 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a1190-2084-4b75-86b3-9bff1bb92d81 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:23.057599 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dcaecf-36ca-4df6-8bca-0fc934c184f9 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:23.099451 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:23.130681 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92836afe-df80-4e70-8e5e-45d84f99acfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:23.131321 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92836afe-df80-4e70-8e5e-45d84f99acfc None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:23.131595 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92836afe-df80-4e70-8e5e-45d84f99acfc None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:23.131860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92836afe-df80-4e70-8e5e-45d84f99acfc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:23.158026 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92836afe-df80-4e70-8e5e-45d84f99acfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:23.163398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-92836afe-df80-4e70-8e5e-45d84f99acfc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:23.163398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:23.163398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:23.163398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:23.163398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:23.163398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:23.163398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:23.163398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:23.163398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:23.163398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:23.163398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:23.163398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:23.163398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:23.163398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:23.163398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:23.163398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:23.164256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:23.164256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:23.164256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:23.164256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:23.164256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:23.164256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:23.164256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:23.164256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:23.164256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:23.164256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:23.164256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:23.164256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:23.164256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:23.164256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:23.164256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:23.164256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:23.165047 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:23.165047 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:23.165047 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:23.165047 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:23.165047 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:23.165047 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:23.165047 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:23.165047 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:23.165047 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:23.165047 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:23.165047 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:23.165047 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:23.165047 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 152/288] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:23.171282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92836afe-df80-4e70-8e5e-45d84f99acfc None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:23.200386 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:23.222870 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a1190-2084-4b75-86b3-9bff1bb92d81 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:23.297917 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2dc63-9729-47dd-b6f7-43efbb362ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:23.298634 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2dc63-9729-47dd-b6f7-43efbb362ed9 None admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:23.298920 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2dc63-9729-47dd-b6f7-43efbb362ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:23.299195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2dc63-9729-47dd-b6f7-43efbb362ed9 None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222/users/7f4bbf36f80d4b92aaaed102c0c6d52a/roles/c4b36aca8aba43d09071f52c763a80a8` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:23.304496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2dc63-9729-47dd-b6f7-43efbb362ed9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e1ab10529d34aaabbf5bdd2cff84222, user_id=7f4bbf36f80d4b92aaaed102c0c6d52a, role_id=c4b36aca8aba43d09071f52c763a80a8)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:23.305982 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2dc63-9729-47dd-b6f7-43efbb362ed9 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:23.328252 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 137/289] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 19:59:23 2022] PUT /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222/users/7f4bbf36f80d4b92aaaed102c0c6d52a/roles/c4b36aca8aba43d09071f52c763a80a8 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:59:23.343490 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92836afe-df80-4e70-8e5e-45d84f99acfc None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:23.390944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d64e2-4d81-4d81-8d93-83a16af66703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:23.391574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d64e2-4d81-4d81-8d93-83a16af66703 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:23.391854 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d64e2-4d81-4d81-8d93-83a16af66703 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:23.392127 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d64e2-4d81-4d81-8d93-83a16af66703 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:23.392866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d64e2-4d81-4d81-8d93-83a16af66703 None admin] RBAC: Adding query filter params (name=member) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:23.393173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d64e2-4d81-4d81-8d93-83a16af66703 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:23.394666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d64e2-4d81-4d81-8d93-83a16af66703 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:23.411136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2dc63-9729-47dd-b6f7-43efbb362ed9 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:23.412188 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 153/290] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:23.564610 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2dc63-9729-47dd-b6f7-43efbb362ed9 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:23.572566 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:23.607756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0d856-5fdb-4f89-bb18-3426b77c46ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:23.608405 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0d856-5fdb-4f89-bb18-3426b77c46ad None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:23.608682 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0d856-5fdb-4f89-bb18-3426b77c46ad None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:23.608953 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0d856-5fdb-4f89-bb18-3426b77c46ad None admin] PATH_INFO: `/v3/users/8394d11508ae4a478697ddaf579deccb` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:23.610149 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0d856-5fdb-4f89-bb18-3426b77c46ad None admin] RBAC: Authorizing `identity:get_user(user_id=8394d11508ae4a478697ddaf579deccb)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:23.611605 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0d856-5fdb-4f89-bb18-3426b77c46ad None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:23.614465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 138/291] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:23 2022] GET /identity/v3/users/8394d11508ae4a478697ddaf579deccb => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:23.619965 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d64e2-4d81-4d81-8d93-83a16af66703 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:23.779932 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:23.796957 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d64e2-4d81-4d81-8d93-83a16af66703 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:23.851930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c3c71-57aa-45ff-83ce-9d100e9c15dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:23.852943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c3c71-57aa-45ff-83ce-9d100e9c15dc None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:23.853421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c3c71-57aa-45ff-83ce-9d100e9c15dc None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:23.853940 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c3c71-57aa-45ff-83ce-9d100e9c15dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:23.947859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c3c71-57aa-45ff-83ce-9d100e9c15dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:23.949938 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c3c71-57aa-45ff-83ce-9d100e9c15dc None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:24.076181 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:24.109353 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 154/292] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:24.115173 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0d856-5fdb-4f89-bb18-3426b77c46ad None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:24.254451 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0d856-5fdb-4f89-bb18-3426b77c46ad None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:24.300325 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a8a70-bbd3-45b2-a962-2854f2b10140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:24.300970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a8a70-bbd3-45b2-a962-2854f2b10140 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:24.301246 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a8a70-bbd3-45b2-a962-2854f2b10140 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:24.301515 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a8a70-bbd3-45b2-a962-2854f2b10140 None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:24.304740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a8a70-bbd3-45b2-a962-2854f2b10140 None admin] RBAC: Authorizing `identity:get_project(project_id=2e1ab10529d34aaabbf5bdd2cff84222)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:24.306285 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a8a70-bbd3-45b2-a962-2854f2b10140 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:24.308832 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 139/293] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:24 2022] GET /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:24.311298 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e30e0d-64db-4b94-9652-50b8ebef5602 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:24.311812 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e30e0d-64db-4b94-9652-50b8ebef5602 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:24.312305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e30e0d-64db-4b94-9652-50b8ebef5602 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:24.314169 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 155/294] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:24.314275 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744e9b3-2c22-406a-948c-f6d49918d831 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:24.316567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744e9b3-2c22-406a-948c-f6d49918d831 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:24.316567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744e9b3-2c22-406a-948c-f6d49918d831 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:24.316739 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e30e0d-64db-4b94-9652-50b8ebef5602 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:24.316830 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 140/295] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:59:24.320542 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998e367-dbbd-457c-b539-c2b892de1e29 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:24.320824 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998e367-dbbd-457c-b539-c2b892de1e29 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:24.321094 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998e367-dbbd-457c-b539-c2b892de1e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:24.342444 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-9998e367-dbbd-457c-b539-c2b892de1e29 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:24.343843 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9998e367-dbbd-457c-b539-c2b892de1e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:24.387265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:24.423420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 141/296] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:24.431662 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de6b53-744b-497e-b1a7-2a4ec8063d40 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:24.432094 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de6b53-744b-497e-b1a7-2a4ec8063d40 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:24.432532 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de6b53-744b-497e-b1a7-2a4ec8063d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:24.435888 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e30e0d-64db-4b94-9652-50b8ebef5602 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:24.467829 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-a6de6b53-744b-497e-b1a7-2a4ec8063d40 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:24.469406 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-a6de6b53-744b-497e-b1a7-2a4ec8063d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:24.486757 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11acec-6c36-4219-a1c7-78844ae51d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:24.487553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11acec-6c36-4219-a1c7-78844ae51d4f None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:24.488000 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11acec-6c36-4219-a1c7-78844ae51d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:24.488360 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11acec-6c36-4219-a1c7-78844ae51d4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:24.489190 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11acec-6c36-4219-a1c7-78844ae51d4f None admin] RBAC: Adding query filter params (role.id=9265e1fd617b4c0b8fbf267885394060, scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222, user.id=8394d11508ae4a478697ddaf579deccb) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:24.489599 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11acec-6c36-4219-a1c7-78844ae51d4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:24.491511 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11acec-6c36-4219-a1c7-78844ae51d4f None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:24.499219 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 156/297] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 19:59:24 2022] GET /identity/v3/role_assignments?user.id=8394d11508ae4a478697ddaf579deccb&scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222&role.id=9265e1fd617b4c0b8fbf267885394060 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:24.500434 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0bb4f-eb2b-4ff4-bf90-2788b2301c8d None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:24.500784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0bb4f-eb2b-4ff4-bf90-2788b2301c8d None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:24.501130 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0bb4f-eb2b-4ff4-bf90-2788b2301c8d None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:24.507049 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 157/298] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:24.508379 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a5fb0-dc1a-43e1-8472-076b76454c21 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:24.508797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a5fb0-dc1a-43e1-8472-076b76454c21 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:24.509075 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a5fb0-dc1a-43e1-8472-076b76454c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:24.528801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-263a5fb0-dc1a-43e1-8472-076b76454c21 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:24.530182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-263a5fb0-dc1a-43e1-8472-076b76454c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:24.573094 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 142/299] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:24.616919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:24.646832 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 158/300] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:24.798876 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de6b53-744b-497e-b1a7-2a4ec8063d40 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:24.884228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a5fb0-dc1a-43e1-8472-076b76454c21 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:24.924963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de6b53-744b-497e-b1a7-2a4ec8063d40 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:24.926898 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:24.965751 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552e664-f661-49f8-8007-c0428f2fb56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:24.966634 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552e664-f661-49f8-8007-c0428f2fb56e None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:24.967094 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552e664-f661-49f8-8007-c0428f2fb56e None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:24.967543 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552e664-f661-49f8-8007-c0428f2fb56e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:24.968704 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552e664-f661-49f8-8007-c0428f2fb56e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:24.970196 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552e664-f661-49f8-8007-c0428f2fb56e None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:24.972083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 143/301] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:24.972871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552e664-f661-49f8-8007-c0428f2fb56e None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:25.016958 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a5fb0-dc1a-43e1-8472-076b76454c21 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:25.055038 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d904c90f-ccca-4d96-9ce4-d0684ff16eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:25.055598 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904c90f-ccca-4d96-9ce4-d0684ff16eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:25.055879 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904c90f-ccca-4d96-9ce4-d0684ff16eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:25.056152 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904c90f-ccca-4d96-9ce4-d0684ff16eb7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:25.068232 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d904c90f-ccca-4d96-9ce4-d0684ff16eb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:25.073560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-d904c90f-ccca-4d96-9ce4-d0684ff16eb7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:25.073560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:25.073560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:25.073560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:25.073560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:25.073560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:25.073560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:25.073560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:25.073560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:25.073560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:25.073560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:25.073560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:25.073560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:25.073560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:25.073560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:25.073560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:25.075432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:25.075432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:25.075432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:25.075432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:25.075432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:25.075432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:25.075432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:25.075432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:25.075432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:25.075432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:25.075432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:25.075432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:25.075432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:25.075432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:25.075432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:25.075432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:25.076221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:25.076221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:25.076221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:25.076221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:25.076221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:25.076221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:25.076221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:25.076221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:25.076221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:25.076221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:25.076221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:25.076221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:25.080375 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 159/302] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:25.081154 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d904c90f-ccca-4d96-9ce4-d0684ff16eb7 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:25.092936 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552e664-f661-49f8-8007-c0428f2fb56e None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:25.131757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed1425-f9c1-45f2-b84e-42314f21552d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:25.132577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed1425-f9c1-45f2-b84e-42314f21552d None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:25.132667 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed1425-f9c1-45f2-b84e-42314f21552d None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:25.132898 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed1425-f9c1-45f2-b84e-42314f21552d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:25.137322 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:25.145183 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed1425-f9c1-45f2-b84e-42314f21552d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:25.150260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-c8ed1425-f9c1-45f2-b84e-42314f21552d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 19:59:25.150260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:25.150260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:25.150260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:25.150260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:25.150260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:25.150260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:25.150260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:25.150260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:25.150260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:25.150260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:25.150260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:25.150260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:25.150260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:25.150260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:25.150260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:25.151270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:25.151270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:25.151270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:25.151270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:25.151270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:25.151270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:25.151270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:25.151270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:25.151270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:25.151270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:25.151270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:25.151270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:25.151270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:25.151270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:25.151270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:25.151270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:25.152120 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:25.152120 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:25.152120 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:25.152120 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:25.152120 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:25.152120 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:25.152120 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:25.152120 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:25.152120 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:25.152120 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:25.152120 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 19:59:25.152120 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:25.152120 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 144/303] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:25.154134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed1425-f9c1-45f2-b84e-42314f21552d None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:25.184438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:25.215811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d904c90f-ccca-4d96-9ce4-d0684ff16eb7 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:25.254734 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c502e-fcf8-4d88-b039-d1b5dcb7da0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:25.255581 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c502e-fcf8-4d88-b039-d1b5dcb7da0d None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:25.256159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c502e-fcf8-4d88-b039-d1b5dcb7da0d None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:25.256441 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c502e-fcf8-4d88-b039-d1b5dcb7da0d None admin] PATH_INFO: `/v3/groups` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:25.257066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c502e-fcf8-4d88-b039-d1b5dcb7da0d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:25.258621 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c502e-fcf8-4d88-b039-d1b5dcb7da0d None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:25.262219 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed1425-f9c1-45f2-b84e-42314f21552d None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:25.285802 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 160/304] 10.222.0.73 () {60 vars in 1134 bytes} [Mon Aug 29 19:59:25 2022] POST /identity/v3/groups => generated 229 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:59:25.285802 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c502e-fcf8-4d88-b039-d1b5dcb7da0d None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:25.302535 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa610767-946a-4d53-a4d9-0ea9ea142431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:25.303183 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa610767-946a-4d53-a4d9-0ea9ea142431 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:25.303461 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa610767-946a-4d53-a4d9-0ea9ea142431 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:25.303728 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa610767-946a-4d53-a4d9-0ea9ea142431 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:25.304409 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa610767-946a-4d53-a4d9-0ea9ea142431 None admin] RBAC: Adding query filter params (name=member) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:25.304711 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa610767-946a-4d53-a4d9-0ea9ea142431 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:25.306462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa610767-946a-4d53-a4d9-0ea9ea142431 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:25.317895 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 145/305] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:25.327943 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa610767-946a-4d53-a4d9-0ea9ea142431 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:25.414840 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:25.426085 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c502e-fcf8-4d88-b039-d1b5dcb7da0d None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:25.458573 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:25.471045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa610767-946a-4d53-a4d9-0ea9ea142431 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:25.472522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7b99f-a582-4507-af05-0e992ae7f7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:25.473126 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7b99f-a582-4507-af05-0e992ae7f7f2 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:25.474249 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7b99f-a582-4507-af05-0e992ae7f7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:25.474249 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7b99f-a582-4507-af05-0e992ae7f7f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:25.474489 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf7b99f-a582-4507-af05-0e992ae7f7f2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:25.474895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf7b99f-a582-4507-af05-0e992ae7f7f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:25.476425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf7b99f-a582-4507-af05-0e992ae7f7f2 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:25.487742 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 161/306] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:25.495195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7b99f-a582-4507-af05-0e992ae7f7f2 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:25.514575 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb35313-2e46-44ef-9814-c755e6d6b752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:25.515220 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb35313-2e46-44ef-9814-c755e6d6b752 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:25.515499 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb35313-2e46-44ef-9814-c755e6d6b752 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:25.515775 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb35313-2e46-44ef-9814-c755e6d6b752 None admin] PATH_INFO: `/v3/users/37beb7365bf449d78a8f7ff0c5a996f0` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:25.516940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb35313-2e46-44ef-9814-c755e6d6b752 None admin] RBAC: Authorizing `identity:get_user(user_id=37beb7365bf449d78a8f7ff0c5a996f0)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:25.518500 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb35313-2e46-44ef-9814-c755e6d6b752 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:25.520465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 146/307] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:25 2022] GET /identity/v3/users/37beb7365bf449d78a8f7ff0c5a996f0 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:25.525937 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb35313-2e46-44ef-9814-c755e6d6b752 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:25.616681 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7b99f-a582-4507-af05-0e992ae7f7f2 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:25.648899 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb35313-2e46-44ef-9814-c755e6d6b752 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:25.655127 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb3d7d-9c0b-4a59-9e85-2e96beb4a4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:25.655901 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb3d7d-9c0b-4a59-9e85-2e96beb4a4cd None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:25.656282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb3d7d-9c0b-4a59-9e85-2e96beb4a4cd None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:25.656656 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb3d7d-9c0b-4a59-9e85-2e96beb4a4cd None admin] PATH_INFO: `/v3/users/76de9ca236aa4cce8ce8b64119c603a7` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:25.671565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb3d7d-9c0b-4a59-9e85-2e96beb4a4cd None admin] RBAC: Authorizing `identity:get_user(user_id=76de9ca236aa4cce8ce8b64119c603a7)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:25.673500 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb3d7d-9c0b-4a59-9e85-2e96beb4a4cd None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:25.675813 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 162/308] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:25 2022] GET /identity/v3/users/76de9ca236aa4cce8ce8b64119c603a7 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:25.680909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb3d7d-9c0b-4a59-9e85-2e96beb4a4cd None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:25.695118 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9a506-67f4-46c5-b5cc-add66880a733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:25.696537 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9a506-67f4-46c5-b5cc-add66880a733 None admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:25.697310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9a506-67f4-46c5-b5cc-add66880a733 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:25.697920 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9a506-67f4-46c5-b5cc-add66880a733 None admin] PATH_INFO: `/v3/system/users/37beb7365bf449d78a8f7ff0c5a996f0/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:25.701181 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9a506-67f4-46c5-b5cc-add66880a733 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=37beb7365bf449d78a8f7ff0c5a996f0, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:25.703338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9a506-67f4-46c5-b5cc-add66880a733 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:25.707623 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:25.716493 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 147/309] 10.222.0.73 () {58 vars in 1334 bytes} [Mon Aug 29 19:59:25 2022] PUT /identity/v3/system/users/37beb7365bf449d78a8f7ff0c5a996f0/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:59:25.756987 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e3bc8-8b99-448e-af58-c6de499ed75a None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:25.757334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e3bc8-8b99-448e-af58-c6de499ed75a None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:25.757529 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e3bc8-8b99-448e-af58-c6de499ed75a None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:25.758936 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 148/310] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:59:25.765424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ace85e-6e3f-4954-b3ce-0d414915c098 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:25.765785 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ace85e-6e3f-4954-b3ce-0d414915c098 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:25.766028 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ace85e-6e3f-4954-b3ce-0d414915c098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:25.786429 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-c3ace85e-6e3f-4954-b3ce-0d414915c098 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:25.788127 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c3ace85e-6e3f-4954-b3ce-0d414915c098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:25.810356 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:25.838248 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb3d7d-9c0b-4a59-9e85-2e96beb4a4cd None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:25.888711 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf1042-11fc-4efe-b8ef-ffa70e36f685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:25.889256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf1042-11fc-4efe-b8ef-ffa70e36f685 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:25.889563 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf1042-11fc-4efe-b8ef-ffa70e36f685 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:25.890003 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf1042-11fc-4efe-b8ef-ffa70e36f685 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:25.890742 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf1042-11fc-4efe-b8ef-ffa70e36f685 None admin] RBAC: Adding query filter params (role.id=c4b36aca8aba43d09071f52c763a80a8, user.id=76de9ca236aa4cce8ce8b64119c603a7, scope.system=all) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:25.891064 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf1042-11fc-4efe-b8ef-ffa70e36f685 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:25.892499 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf1042-11fc-4efe-b8ef-ffa70e36f685 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:25.894815 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 149/311] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:25.901781 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 163/312] 10.222.0.73 () {56 vars in 1308 bytes} [Mon Aug 29 19:59:25 2022] GET /identity/v3/role_assignments?user.id=76de9ca236aa4cce8ce8b64119c603a7&scope.system=all&role.id=c4b36aca8aba43d09071f52c763a80a8 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:26.177581 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ace85e-6e3f-4954-b3ce-0d414915c098 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:26.284618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ace85e-6e3f-4954-b3ce-0d414915c098 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:26.321117 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc637d9f-5180-4f79-901a-e9828560d712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:26.322190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc637d9f-5180-4f79-901a-e9828560d712 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:26.322675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc637d9f-5180-4f79-901a-e9828560d712 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:26.323173 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc637d9f-5180-4f79-901a-e9828560d712 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:26.334662 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc637d9f-5180-4f79-901a-e9828560d712 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:26.340471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-dc637d9f-5180-4f79-901a-e9828560d712 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 19:59:26.340471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:26.340471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:26.340471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:26.340471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:26.340471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:26.340471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:26.340471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:26.340471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:26.340471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:26.340471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:26.340471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:26.340471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:26.340471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:26.340471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:26.340471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:26.341332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:26.341332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:26.341332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:26.341332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:26.341332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:26.341332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:26.341332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:26.341332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:26.341332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:26.341332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:26.341332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:26.341332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:26.341332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:26.341332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:26.341332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:26.341332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:26.342186 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:26.342186 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:26.342186 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:26.342186 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:26.342186 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:26.342186 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:26.342186 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:26.342186 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:26.342186 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:26.342186 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:26.342186 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 19:59:26.342186 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:26.344524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 150/313] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:26.350744 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf1042-11fc-4efe-b8ef-ffa70e36f685 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:26.445700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:26.470082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf1042-11fc-4efe-b8ef-ffa70e36f685 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:26.508406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e5349-9620-42bd-bf70-5fc38f366db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:26.509014 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e5349-9620-42bd-bf70-5fc38f366db2 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:26.509295 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e5349-9620-42bd-bf70-5fc38f366db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:26.509564 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e5349-9620-42bd-bf70-5fc38f366db2 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:26.510261 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4e5349-9620-42bd-bf70-5fc38f366db2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:26.510562 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4e5349-9620-42bd-bf70-5fc38f366db2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:26.512060 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4e5349-9620-42bd-bf70-5fc38f366db2 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:26.520710 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 164/314] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:26.525978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc637d9f-5180-4f79-901a-e9828560d712 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:26.543495 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:26.632465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc637d9f-5180-4f79-901a-e9828560d712 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:26.672699 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3796eb-a881-4401-83d4-e77f53e20850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:26.673382 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3796eb-a881-4401-83d4-e77f53e20850 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:26.673747 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3796eb-a881-4401-83d4-e77f53e20850 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:26.674032 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3796eb-a881-4401-83d4-e77f53e20850 None admin] PATH_INFO: `/v3/users/7f4bbf36f80d4b92aaaed102c0c6d52a` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:26.675183 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3796eb-a881-4401-83d4-e77f53e20850 None admin] RBAC: Authorizing `identity:get_user(user_id=7f4bbf36f80d4b92aaaed102c0c6d52a)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:26.676693 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3796eb-a881-4401-83d4-e77f53e20850 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:26.678667 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 151/315] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:26 2022] GET /identity/v3/users/7f4bbf36f80d4b92aaaed102c0c6d52a => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:26.684718 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e5349-9620-42bd-bf70-5fc38f366db2 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:26.796344 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e5349-9620-42bd-bf70-5fc38f366db2 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:26.832720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906ac48-0df2-4384-a126-e0c31c95b21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:26.833343 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906ac48-0df2-4384-a126-e0c31c95b21f None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:26.833624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906ac48-0df2-4384-a126-e0c31c95b21f None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:26.833974 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906ac48-0df2-4384-a126-e0c31c95b21f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:26.850962 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:26.907958 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906ac48-0df2-4384-a126-e0c31c95b21f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:26.909421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906ac48-0df2-4384-a126-e0c31c95b21f None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:27.019114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 165/316] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:27.024412 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3796eb-a881-4401-83d4-e77f53e20850 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:27.119817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:27.138990 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3796eb-a881-4401-83d4-e77f53e20850 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:27.180590 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7dc0e-e8c9-427c-ac8d-35b76ba27464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:27.182059 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7dc0e-e8c9-427c-ac8d-35b76ba27464 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:27.182570 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7dc0e-e8c9-427c-ac8d-35b76ba27464 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:27.183042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7dc0e-e8c9-427c-ac8d-35b76ba27464 None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:27.184482 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e7dc0e-e8c9-427c-ac8d-35b76ba27464 None admin] RBAC: Authorizing `identity:get_project(project_id=2e1ab10529d34aaabbf5bdd2cff84222)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:27.186174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e7dc0e-e8c9-427c-ac8d-35b76ba27464 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:27.188319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 152/317] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:27 2022] GET /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:27.194086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906ac48-0df2-4384-a126-e0c31c95b21f None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:27.250484 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:27.308730 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906ac48-0df2-4384-a126-e0c31c95b21f None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:27.320360 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b42102-75ca-4314-99ff-94269d4d8c58 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:27.320645 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b42102-75ca-4314-99ff-94269d4d8c58 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:27.320907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b42102-75ca-4314-99ff-94269d4d8c58 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:27.322289 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 153/318] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:59:27.327727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe06a24-9004-4b18-867c-c4b4a47157f9 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:27.328011 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe06a24-9004-4b18-867c-c4b4a47157f9 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:27.328286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe06a24-9004-4b18-867c-c4b4a47157f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:27.345629 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed3e1f-351c-42b8-b4db-c639df99c6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:27.346202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed3e1f-351c-42b8-b4db-c639df99c6bd None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:27.346486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed3e1f-351c-42b8-b4db-c639df99c6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:27.346757 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed3e1f-351c-42b8-b4db-c639df99c6bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:27.347490 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed3e1f-351c-42b8-b4db-c639df99c6bd None admin] RBAC: Adding query filter params (role.id=c4b36aca8aba43d09071f52c763a80a8, scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222, user.id=7f4bbf36f80d4b92aaaed102c0c6d52a) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:27.347796 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed3e1f-351c-42b8-b4db-c639df99c6bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:27.349234 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed3e1f-351c-42b8-b4db-c639df99c6bd None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:27.351833 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-cbe06a24-9004-4b18-867c-c4b4a47157f9 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:27.353976 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-cbe06a24-9004-4b18-867c-c4b4a47157f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:27.358705 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 166/319] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 19:59:27 2022] GET /identity/v3/role_assignments?user.id=7f4bbf36f80d4b92aaaed102c0c6d52a&scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222&role.id=c4b36aca8aba43d09071f52c763a80a8 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:27.440195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 154/320] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:59:27.712323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed3e1f-351c-42b8-b4db-c639df99c6bd None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:27.819083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed3e1f-351c-42b8-b4db-c639df99c6bd None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:27.830407 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:27.848839 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4d58b-e146-44c6-b9ae-2ac04876d0d0 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:27.849143 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4d58b-e146-44c6-b9ae-2ac04876d0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:27.849404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4d58b-e146-44c6-b9ae-2ac04876d0d0 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:27.850912 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 155/321] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:59:27.856392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d041adc-1b3f-447a-b41d-93d110bbc1d9 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:27.856895 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d041adc-1b3f-447a-b41d-93d110bbc1d9 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:27.857172 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d041adc-1b3f-447a-b41d-93d110bbc1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:27.863705 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b27f9-0f14-46c6-b003-f7260f23ef78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:27.864344 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b27f9-0f14-46c6-b003-f7260f23ef78 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:27.864624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b27f9-0f14-46c6-b003-f7260f23ef78 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:27.864897 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b27f9-0f14-46c6-b003-f7260f23ef78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:27.866095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b27f9-0f14-46c6-b003-f7260f23ef78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:27.867539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b27f9-0f14-46c6-b003-f7260f23ef78 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:27.869277 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 167/322] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:27.875460 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b27f9-0f14-46c6-b003-f7260f23ef78 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:27.877524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-9d041adc-1b3f-447a-b41d-93d110bbc1d9 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:27.878746 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9d041adc-1b3f-447a-b41d-93d110bbc1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:27.954323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 156/323] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:59:27.955388 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a64f1-4c7b-4d96-a268-c4338591c3e8 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:27.955673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a64f1-4c7b-4d96-a268-c4338591c3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:27.955944 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a64f1-4c7b-4d96-a268-c4338591c3e8 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:27.957185 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 157/324] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:27.964774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f700c-d216-4dd3-b649-e4179929a0b4 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:27.964774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f700c-d216-4dd3-b649-e4179929a0b4 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:27.964774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f700c-d216-4dd3-b649-e4179929a0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:27.983656 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-d00f700c-d216-4dd3-b649-e4179929a0b4 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:27.984837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d00f700c-d216-4dd3-b649-e4179929a0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:27.987706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:27.992903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b27f9-0f14-46c6-b003-f7260f23ef78 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:28.029460 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182ac55-689e-4e08-a0a6-d6d6b3b3f6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:28.030116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182ac55-689e-4e08-a0a6-d6d6b3b3f6e9 None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:28.030395 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182ac55-689e-4e08-a0a6-d6d6b3b3f6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:28.030666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182ac55-689e-4e08-a0a6-d6d6b3b3f6e9 None admin] PATH_INFO: `/v3/groups` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:28.031298 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182ac55-689e-4e08-a0a6-d6d6b3b3f6e9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:28.032729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182ac55-689e-4e08-a0a6-d6d6b3b3f6e9 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:28.050584 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 168/325] 10.222.0.73 () {60 vars in 1134 bytes} [Mon Aug 29 19:59:27 2022] POST /identity/v3/groups => generated 226 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:59:28.065109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 158/326] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:59:28.222917 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182ac55-689e-4e08-a0a6-d6d6b3b3f6e9 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:28.269331 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:28.335826 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f700c-d216-4dd3-b649-e4179929a0b4 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:28.339125 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182ac55-689e-4e08-a0a6-d6d6b3b3f6e9 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:28.388153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c035035-9a58-4e2c-9317-0fc8b262ab44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:28.388904 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c035035-9a58-4e2c-9317-0fc8b262ab44 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:28.389090 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c035035-9a58-4e2c-9317-0fc8b262ab44 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:28.389332 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c035035-9a58-4e2c-9317-0fc8b262ab44 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:28.400676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c035035-9a58-4e2c-9317-0fc8b262ab44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:28.406196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-8c035035-9a58-4e2c-9317-0fc8b262ab44 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:28.406196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:28.406196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:28.406196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:28.406196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:28.406196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:28.406196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:28.406196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:28.406196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:28.406196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:28.406196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:28.406196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:28.406196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:28.406196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:28.406196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:28.406196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:28.407100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:28.407100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:28.407100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:28.407100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:28.407100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:28.407100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:28.407100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:28.407100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:28.407100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:28.407100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:28.407100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:28.407100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:28.407100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:28.407100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:28.407100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:28.407100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:28.407908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:28.407908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:28.407908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:28.407908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:28.407908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:28.407908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:28.407908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:28.407908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:28.407908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:28.407908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:28.407908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:28.407908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:28.408795 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 169/327] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:28.413412 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c035035-9a58-4e2c-9317-0fc8b262ab44 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:28.475262 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f700c-d216-4dd3-b649-e4179929a0b4 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:28.522709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331cfae-6f87-47e4-93e1-a6a43b5d75b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:28.523901 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331cfae-6f87-47e4-93e1-a6a43b5d75b7 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:28.524180 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331cfae-6f87-47e4-93e1-a6a43b5d75b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:28.524449 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331cfae-6f87-47e4-93e1-a6a43b5d75b7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:28.538704 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:28.541803 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1331cfae-6f87-47e4-93e1-a6a43b5d75b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:28.548710 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:28.549134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-1331cfae-6f87-47e4-93e1-a6a43b5d75b7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 19:59:28.549134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:28.549134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:28.549134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:28.549134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:28.549134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:28.549134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:28.549134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:28.549134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:28.549134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:28.549134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:28.549134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:28.549134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:28.549134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:28.549134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:28.549134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:28.550026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:28.550026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:28.550026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:28.550026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:28.550026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:28.550026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:28.550026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:28.550026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:28.550026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:28.550026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:28.550026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:28.550026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:28.550026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:28.550026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:28.550026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:28.550026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:28.550807 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:28.550807 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:28.550807 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:28.550807 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:28.550807 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:28.550807 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:28.550807 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:28.550807 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:28.550807 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:28.550807 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:28.550807 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 19:59:28.550807 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:28.551403 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 159/328] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 19:59:28.556836 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c035035-9a58-4e2c-9317-0fc8b262ab44 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:28.557271 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331cfae-6f87-47e4-93e1-a6a43b5d75b7 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:28.603226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37880ebb-dedc-45a0-9ab6-423243005c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:28.604226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37880ebb-dedc-45a0-9ab6-423243005c31 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:28.604718 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37880ebb-dedc-45a0-9ab6-423243005c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:28.605181 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37880ebb-dedc-45a0-9ab6-423243005c31 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:28.606968 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37880ebb-dedc-45a0-9ab6-423243005c31 None admin] RBAC: Adding query filter params (name=member) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:28.607583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37880ebb-dedc-45a0-9ab6-423243005c31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:28.609301 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37880ebb-dedc-45a0-9ab6-423243005c31 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:28.620480 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 170/329] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:28.628172 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37880ebb-dedc-45a0-9ab6-423243005c31 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:28.699524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331cfae-6f87-47e4-93e1-a6a43b5d75b7 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:28.738720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff46e78-6c8b-4ca5-93e2-284b7aeca0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:28.739742 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff46e78-6c8b-4ca5-93e2-284b7aeca0d7 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:28.740216 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff46e78-6c8b-4ca5-93e2-284b7aeca0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:28.740681 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff46e78-6c8b-4ca5-93e2-284b7aeca0d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:28.741537 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff46e78-6c8b-4ca5-93e2-284b7aeca0d7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:28.742064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff46e78-6c8b-4ca5-93e2-284b7aeca0d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:28.744420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff46e78-6c8b-4ca5-93e2-284b7aeca0d7 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:28.755653 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 160/330] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:28.764075 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff46e78-6c8b-4ca5-93e2-284b7aeca0d7 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:28.784122 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37880ebb-dedc-45a0-9ab6-423243005c31 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:28.826674 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bce67-4614-4b49-addf-968c8e400445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:28.827744 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bce67-4614-4b49-addf-968c8e400445 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:28.828240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bce67-4614-4b49-addf-968c8e400445 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:28.828707 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bce67-4614-4b49-addf-968c8e400445 None admin] PATH_INFO: `/v3/users/37beb7365bf449d78a8f7ff0c5a996f0` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:28.830162 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1bce67-4614-4b49-addf-968c8e400445 None admin] RBAC: Authorizing `identity:get_user(user_id=37beb7365bf449d78a8f7ff0c5a996f0)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:28.831891 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1bce67-4614-4b49-addf-968c8e400445 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:28.834106 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 171/331] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:28 2022] GET /identity/v3/users/37beb7365bf449d78a8f7ff0c5a996f0 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:28.840979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bce67-4614-4b49-addf-968c8e400445 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:28.854132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:28.876656 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:28.923267 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff46e78-6c8b-4ca5-93e2-284b7aeca0d7 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:28.957831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bce67-4614-4b49-addf-968c8e400445 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:28.960255 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f8dc0-3fa5-4e39-8640-9f81d01090c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:28.960899 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f8dc0-3fa5-4e39-8640-9f81d01090c9 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:28.961180 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f8dc0-3fa5-4e39-8640-9f81d01090c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:28.961449 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f8dc0-3fa5-4e39-8640-9f81d01090c9 None admin] PATH_INFO: `/v3/users/76de9ca236aa4cce8ce8b64119c603a7` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:28.962648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f8dc0-3fa5-4e39-8640-9f81d01090c9 None admin] RBAC: Authorizing `identity:get_user(user_id=76de9ca236aa4cce8ce8b64119c603a7)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:28.964083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f8dc0-3fa5-4e39-8640-9f81d01090c9 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:28.966932 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 161/332] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:28 2022] GET /identity/v3/users/76de9ca236aa4cce8ce8b64119c603a7 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:28.973374 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f8dc0-3fa5-4e39-8640-9f81d01090c9 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:28.999394 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9d88f-5901-4c4a-8490-e2d893d57f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:28.999935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9d88f-5901-4c4a-8490-e2d893d57f4e None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:29.000210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9d88f-5901-4c4a-8490-e2d893d57f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:29.000475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9d88f-5901-4c4a-8490-e2d893d57f4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:29.001197 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9d88f-5901-4c4a-8490-e2d893d57f4e None admin] RBAC: Adding query filter params (role.id=9265e1fd617b4c0b8fbf267885394060, user.id=37beb7365bf449d78a8f7ff0c5a996f0, scope.system=all) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:29.001500 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9d88f-5901-4c4a-8490-e2d893d57f4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:29.002985 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9d88f-5901-4c4a-8490-e2d893d57f4e None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:29.017367 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 172/333] 10.222.0.73 () {56 vars in 1308 bytes} [Mon Aug 29 19:59:28 2022] GET /identity/v3/role_assignments?user.id=37beb7365bf449d78a8f7ff0c5a996f0&scope.system=all&role.id=9265e1fd617b4c0b8fbf267885394060 => generated 510 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:29.105314 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f8dc0-3fa5-4e39-8640-9f81d01090c9 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:29.139860 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:29.147256 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00687878-73d1-4c86-b240-e1e7cb74ad3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:29.147993 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00687878-73d1-4c86-b240-e1e7cb74ad3b None admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:29.148268 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00687878-73d1-4c86-b240-e1e7cb74ad3b None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:29.148613 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00687878-73d1-4c86-b240-e1e7cb74ad3b None admin] PATH_INFO: `/v3/system/users/76de9ca236aa4cce8ce8b64119c603a7/roles/c4b36aca8aba43d09071f52c763a80a8` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:29.150205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00687878-73d1-4c86-b240-e1e7cb74ad3b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=76de9ca236aa4cce8ce8b64119c603a7, role_id=c4b36aca8aba43d09071f52c763a80a8)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:29.151599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00687878-73d1-4c86-b240-e1e7cb74ad3b None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:29.161882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 162/334] 10.222.0.73 () {58 vars in 1334 bytes} [Mon Aug 29 19:59:28 2022] PUT /identity/v3/system/users/76de9ca236aa4cce8ce8b64119c603a7/roles/c4b36aca8aba43d09071f52c763a80a8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:59:30.516764 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e9a30-0966-450b-a6c8-0063335e59db None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:30.517431 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e9a30-0966-450b-a6c8-0063335e59db None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:30.517431 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e9a30-0966-450b-a6c8-0063335e59db None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:30.518915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 173/335] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:30.524666 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7280c1c-c32f-460e-a05e-9beef443f3a5 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:30.525318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7280c1c-c32f-460e-a05e-9beef443f3a5 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:30.525318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7280c1c-c32f-460e-a05e-9beef443f3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:30.550285 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d19ff-d945-45db-a949-2ea1a86dbb95 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:30.550614 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d19ff-d945-45db-a949-2ea1a86dbb95 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:30.550814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d19ff-d945-45db-a949-2ea1a86dbb95 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:30.552457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 174/336] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:59:30.553360 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-b7280c1c-c32f-460e-a05e-9beef443f3a5 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:30.554634 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-b7280c1c-c32f-460e-a05e-9beef443f3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:30.580516 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ce0b1-b361-4fc6-b6d7-3b2695a20490 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:30.580818 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ce0b1-b361-4fc6-b6d7-3b2695a20490 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:30.581048 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ce0b1-b361-4fc6-b6d7-3b2695a20490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:30.607429 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-994ce0b1-b361-4fc6-b6d7-3b2695a20490 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:30.608622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-994ce0b1-b361-4fc6-b6d7-3b2695a20490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:30.678085 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 163/337] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:30.715796 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 175/338] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:59:31.221339 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7280c1c-c32f-460e-a05e-9beef443f3a5 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:31.249145 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ce0b1-b361-4fc6-b6d7-3b2695a20490 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:31.271819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:31.420994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7280c1c-c32f-460e-a05e-9beef443f3a5 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:31.433657 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ce0b1-b361-4fc6-b6d7-3b2695a20490 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:31.467353 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:31.489941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca81c9c-602f-4bd4-bb43-1228e6377153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:31.491430 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca81c9c-602f-4bd4-bb43-1228e6377153 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:31.492306 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca81c9c-602f-4bd4-bb43-1228e6377153 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:31.492890 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca81c9c-602f-4bd4-bb43-1228e6377153 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:31.503082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1839f-8426-4aba-bb0d-9c038b789788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:31.503949 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1839f-8426-4aba-bb0d-9c038b789788 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:31.504366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1839f-8426-4aba-bb0d-9c038b789788 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:31.504795 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1839f-8426-4aba-bb0d-9c038b789788 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:31.521407 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca81c9c-602f-4bd4-bb43-1228e6377153 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:31.526928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-8ca81c9c-602f-4bd4-bb43-1228e6377153 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:31.526928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:31.526928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:31.526928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:31.526928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:31.526928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:31.526928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:31.526928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:31.526928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:31.526928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:31.526928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:31.526928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:31.526928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:31.526928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:31.526928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:31.526928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:31.528265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:31.528265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:31.528265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:31.528265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:31.528265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:31.528265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:31.528265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:31.528265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:31.528265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:31.528265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:31.528265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:31.528265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:31.528265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:31.528265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:31.528265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:31.528265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:31.529045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:31.529045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:31.529045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:31.529045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:31.529045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:31.529045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:31.529045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:31.529045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:31.529045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:31.529045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:31.529045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:31.529045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:31.529045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 164/339] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:31.533296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa1839f-8426-4aba-bb0d-9c038b789788 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:31.534600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca81c9c-602f-4bd4-bb43-1228e6377153 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:31.543769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-4fa1839f-8426-4aba-bb0d-9c038b789788 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:31.543769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:31.543769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:31.543769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:31.543769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:31.543769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:31.543769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:31.543769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:31.543769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:31.543769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:31.543769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:31.543769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:31.543769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:31.543769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:31.543769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:31.543769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:31.544659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:31.544659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:31.544659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:31.544659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:31.544659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:31.544659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:31.544659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:31.544659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:31.544659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:31.544659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:31.544659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:31.544659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:31.544659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:31.544659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:31.544659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:31.544659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:31.545438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:31.545438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:31.545438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:31.545438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:31.545438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:31.545438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:31.545438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:31.545438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:31.545438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:31.545438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:31.545438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:31.545438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:31.548035 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 176/340] 10.222.0.73 () {56 vars in 1114 bytes} [Mon Aug 29 19:59:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:31.553768 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1839f-8426-4aba-bb0d-9c038b789788 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:31.702168 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:31.741298 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca81c9c-602f-4bd4-bb43-1228e6377153 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:31.743801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1839f-8426-4aba-bb0d-9c038b789788 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:31.798976 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce575488-f31f-4130-bc8e-88a6c1fa06ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:31.799689 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce575488-f31f-4130-bc8e-88a6c1fa06ac None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:31.799893 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce575488-f31f-4130-bc8e-88a6c1fa06ac None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:31.800135 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce575488-f31f-4130-bc8e-88a6c1fa06ac None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:31.800819 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce575488-f31f-4130-bc8e-88a6c1fa06ac None admin] RBAC: Adding query filter params (name=member) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:31.801107 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce575488-f31f-4130-bc8e-88a6c1fa06ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:31.802509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce575488-f31f-4130-bc8e-88a6c1fa06ac None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:31.814869 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 165/341] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:31.815858 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccb8d9-311c-4386-9992-f5a25a8e06dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:31.816206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27c5be-f3c9-4780-b011-2c4ef1fdbc6e None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:31.816494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27c5be-f3c9-4780-b011-2c4ef1fdbc6e None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:31.816765 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27c5be-f3c9-4780-b011-2c4ef1fdbc6e None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:31.817412 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccb8d9-311c-4386-9992-f5a25a8e06dc None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:31.817912 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccb8d9-311c-4386-9992-f5a25a8e06dc None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:31.818248 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 166/342] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:31.818571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccb8d9-311c-4386-9992-f5a25a8e06dc None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:31.819779 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fccb8d9-311c-4386-9992-f5a25a8e06dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:31.819779 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fccb8d9-311c-4386-9992-f5a25a8e06dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:31.821218 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fccb8d9-311c-4386-9992-f5a25a8e06dc None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:31.824325 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609831b-fa54-4b64-b253-6e99118b4fed None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:31.825328 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609831b-fa54-4b64-b253-6e99118b4fed None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:31.825913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609831b-fa54-4b64-b253-6e99118b4fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:31.836416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 177/343] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 19:59:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:31.837370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccb8d9-311c-4386-9992-f5a25a8e06dc None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:31.847835 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-a609831b-fa54-4b64-b253-6e99118b4fed None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:31.849318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-a609831b-fa54-4b64-b253-6e99118b4fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:31.900482 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:31.929113 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 167/344] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:31.930337 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609831b-fa54-4b64-b253-6e99118b4fed None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:32.010968 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccb8d9-311c-4386-9992-f5a25a8e06dc None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:32.040997 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609831b-fa54-4b64-b253-6e99118b4fed None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:32.051930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebb916-5db3-492f-bf34-646f8b191382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:32.052694 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebb916-5db3-492f-bf34-646f8b191382 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:32.052854 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebb916-5db3-492f-bf34-646f8b191382 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:32.053098 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebb916-5db3-492f-bf34-646f8b191382 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:32.080417 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1b2d0-276f-401c-929b-a6ea03c8dc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:32.081402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1b2d0-276f-401c-929b-a6ea03c8dc21 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:32.081896 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1b2d0-276f-401c-929b-a6ea03c8dc21 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:32.082342 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1b2d0-276f-401c-929b-a6ea03c8dc21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:32.110115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:32.138437 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:32.155018 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebb916-5db3-492f-bf34-646f8b191382 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:32.156632 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebb916-5db3-492f-bf34-646f8b191382 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:32.165409 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f1b2d0-276f-401c-929b-a6ea03c8dc21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:32.167218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f1b2d0-276f-401c-929b-a6ea03c8dc21 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:32.274721 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 168/345] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:32.276253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fd1b5-2935-461f-b8ee-a30598c412ba None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:32.276713 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fd1b5-2935-461f-b8ee-a30598c412ba None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:32.277176 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fd1b5-2935-461f-b8ee-a30598c412ba None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:32.278760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 169/346] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:32.285648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444a6ff-82ea-4eaf-a747-bd8c03b6bc69 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:32.286350 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444a6ff-82ea-4eaf-a747-bd8c03b6bc69 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:32.286823 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444a6ff-82ea-4eaf-a747-bd8c03b6bc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:32.308187 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-7444a6ff-82ea-4eaf-a747-bd8c03b6bc69 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:32.310487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-7444a6ff-82ea-4eaf-a747-bd8c03b6bc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:32.323862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 178/347] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:32.325234 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebb916-5db3-492f-bf34-646f8b191382 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:32.372866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:32.405807 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 170/348] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:32.407191 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444a6ff-82ea-4eaf-a747-bd8c03b6bc69 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:32.416729 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:32.442992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebb916-5db3-492f-bf34-646f8b191382 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:32.479193 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd911b18-98d6-49cc-bfd6-7fb1df9def0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:32.479829 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd911b18-98d6-49cc-bfd6-7fb1df9def0a None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:32.480113 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd911b18-98d6-49cc-bfd6-7fb1df9def0a None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:32.480386 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd911b18-98d6-49cc-bfd6-7fb1df9def0a None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:32.481622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd911b18-98d6-49cc-bfd6-7fb1df9def0a None admin] RBAC: Authorizing `identity:get_project(project_id=4367ec2ce611409a85975a2a505f1ff8)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:32.483121 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd911b18-98d6-49cc-bfd6-7fb1df9def0a None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:32.484973 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 179/349] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:32 2022] GET /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:32.488332 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd911b18-98d6-49cc-bfd6-7fb1df9def0a None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:32.538371 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444a6ff-82ea-4eaf-a747-bd8c03b6bc69 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:32.592202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd911b18-98d6-49cc-bfd6-7fb1df9def0a None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:32.599654 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:32.610293 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e927fa1-e264-47ec-870c-2ea86fc29ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:32.611425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e927fa1-e264-47ec-870c-2ea86fc29ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:32.611992 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e927fa1-e264-47ec-870c-2ea86fc29ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:32.612454 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e927fa1-e264-47ec-870c-2ea86fc29ab5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:32.630254 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bec68-7246-44cd-a227-b970b30c5ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:32.630909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bec68-7246-44cd-a227-b970b30c5ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:32.631200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bec68-7246-44cd-a227-b970b30c5ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:32.631454 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bec68-7246-44cd-a227-b970b30c5ae3 None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:32.632599 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799bec68-7246-44cd-a227-b970b30c5ae3 None admin] RBAC: Authorizing `identity:get_project(project_id=4367ec2ce611409a85975a2a505f1ff8)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:32.635367 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e927fa1-e264-47ec-870c-2ea86fc29ab5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:32.635485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799bec68-7246-44cd-a227-b970b30c5ae3 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:32.637957 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 180/350] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:32 2022] GET /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:32.638794 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bec68-7246-44cd-a227-b970b30c5ae3 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:32.646323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-5e927fa1-e264-47ec-870c-2ea86fc29ab5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 19:59:32.646323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:32.646323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:32.646323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:32.646323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:32.646323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:32.646323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:32.646323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:32.646323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:32.646323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:32.646323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:32.646323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:32.646323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:32.646323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:32.646323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:32.646323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:32.647256 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:32.647256 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:32.647256 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:32.647256 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:32.647256 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:32.647256 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:32.647256 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:32.647256 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:32.647256 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:32.647256 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:32.647256 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:32.647256 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:32.647256 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:32.647256 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:32.647256 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:32.647256 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:32.648069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:32.648069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:32.648069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:32.648069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:32.648069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:32.648069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:32.648069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:32.648069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:32.648069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:32.648069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:32.648069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 19:59:32.648069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:32.650514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 171/351] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:32.651738 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e927fa1-e264-47ec-870c-2ea86fc29ab5 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:32.780196 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:32.788129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bec68-7246-44cd-a227-b970b30c5ae3 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:32.827602 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e927fa1-e264-47ec-870c-2ea86fc29ab5 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:32.828433 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5dc2e6-8d2c-4f38-bb2b-e0566d1f1fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:32.829083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dc2e6-8d2c-4f38-bb2b-e0566d1f1fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:32.829361 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dc2e6-8d2c-4f38-bb2b-e0566d1f1fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:32.829631 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dc2e6-8d2c-4f38-bb2b-e0566d1f1fc6 None admin] PATH_INFO: `/v3/groups/87f7784d531244e9963fa10fbbf8ed8f` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:32.837189 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5dc2e6-8d2c-4f38-bb2b-e0566d1f1fc6 None admin] RBAC: Authorizing `identity:get_group(group_id=87f7784d531244e9963fa10fbbf8ed8f)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:32.838736 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5dc2e6-8d2c-4f38-bb2b-e0566d1f1fc6 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:32.840474 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 181/352] 10.222.0.73 () {56 vars in 1180 bytes} [Mon Aug 29 19:59:32 2022] GET /identity/v3/groups/87f7784d531244e9963fa10fbbf8ed8f => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:32.841262 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5dc2e6-8d2c-4f38-bb2b-e0566d1f1fc6 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:32.868721 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf74e7-6c36-4f4f-9683-f46e588b3e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:32.869370 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf74e7-6c36-4f4f-9683-f46e588b3e2a None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:32.869652 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf74e7-6c36-4f4f-9683-f46e588b3e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:32.869953 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf74e7-6c36-4f4f-9683-f46e588b3e2a None admin] PATH_INFO: `/v3/groups/87f7784d531244e9963fa10fbbf8ed8f` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:32.871935 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf74e7-6c36-4f4f-9683-f46e588b3e2a None admin] RBAC: Authorizing `identity:get_group(group_id=87f7784d531244e9963fa10fbbf8ed8f)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:32.873391 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf74e7-6c36-4f4f-9683-f46e588b3e2a None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:32.875470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 172/353] 10.222.0.73 () {56 vars in 1180 bytes} [Mon Aug 29 19:59:32 2022] GET /identity/v3/groups/87f7784d531244e9963fa10fbbf8ed8f => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:32.876325 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf74e7-6c36-4f4f-9683-f46e588b3e2a None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:32.888598 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:32.961179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5dc2e6-8d2c-4f38-bb2b-e0566d1f1fc6 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:32.990479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf74e7-6c36-4f4f-9683-f46e588b3e2a None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:33.006900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4797318-5d81-4caf-b1ce-6b5d5a4e3976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:33.007520 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4797318-5d81-4caf-b1ce-6b5d5a4e3976 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:33.007800 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4797318-5d81-4caf-b1ce-6b5d5a4e3976 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:33.008606 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4797318-5d81-4caf-b1ce-6b5d5a4e3976 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:33.008862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4797318-5d81-4caf-b1ce-6b5d5a4e3976 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:33.009173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4797318-5d81-4caf-b1ce-6b5d5a4e3976 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:33.011404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4797318-5d81-4caf-b1ce-6b5d5a4e3976 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:33.021365 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 182/354] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:33.022269 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4797318-5d81-4caf-b1ce-6b5d5a4e3976 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:33.038378 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4ee7c-7979-4dba-96a5-c3a2239643d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:33.039420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4ee7c-7979-4dba-96a5-c3a2239643d6 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:33.039698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4ee7c-7979-4dba-96a5-c3a2239643d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:33.040153 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4ee7c-7979-4dba-96a5-c3a2239643d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:33.042794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:33.056537 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4ee7c-7979-4dba-96a5-c3a2239643d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:33.062001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-a7c4ee7c-7979-4dba-96a5-c3a2239643d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:33.062001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:33.062001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:33.062001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:33.062001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:33.062001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:33.062001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:33.062001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:33.062001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:33.062001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:33.062001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:33.062001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:33.062001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:33.062001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:33.062001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:33.062001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:33.062902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:33.062902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:33.062902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:33.062902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:33.062902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:33.062902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:33.062902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:33.062902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:33.062902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:33.062902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:33.062902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:33.062902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:33.062902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:33.062902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:33.062902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:33.062902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:33.063686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:33.063686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:33.063686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:33.063686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:33.063686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:33.063686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:33.063686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:33.063686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:33.063686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:33.063686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:33.063686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:33.063686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:33.063686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 173/355] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:33.064326 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4ee7c-7979-4dba-96a5-c3a2239643d6 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:33.153356 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:33.157260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4797318-5d81-4caf-b1ce-6b5d5a4e3976 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:33.187439 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4ee7c-7979-4dba-96a5-c3a2239643d6 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:33.199748 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f8ba0-91e1-44ee-92af-e3865d69d1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:33.199748 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f8ba0-91e1-44ee-92af-e3865d69d1a8 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:33.199748 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f8ba0-91e1-44ee-92af-e3865d69d1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:33.199748 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f8ba0-91e1-44ee-92af-e3865d69d1a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:33.200203 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f8ba0-91e1-44ee-92af-e3865d69d1a8 None admin] RBAC: Adding query filter params (group.id=87f7784d531244e9963fa10fbbf8ed8f, role.id=b9184599d95b4950beca34304cdd2eb8, scope.project.id=4367ec2ce611409a85975a2a505f1ff8) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:33.200506 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f8ba0-91e1-44ee-92af-e3865d69d1a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:33.202049 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f8ba0-91e1-44ee-92af-e3865d69d1a8 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:33.209438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 183/356] 10.222.0.73 () {56 vars in 1376 bytes} [Mon Aug 29 19:59:33 2022] GET /identity/v3/role_assignments?group.id=87f7784d531244e9963fa10fbbf8ed8f&scope.project.id=4367ec2ce611409a85975a2a505f1ff8&role.id=b9184599d95b4950beca34304cdd2eb8 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:33.209438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081ef78-9b63-4e67-80f7-7b8baa04e582 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:33.209438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081ef78-9b63-4e67-80f7-7b8baa04e582 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:33.209775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081ef78-9b63-4e67-80f7-7b8baa04e582 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:33.211304 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 184/357] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:33.212008 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d314bb9-dd85-4870-97da-7c8c56eab46d None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:33.212291 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d314bb9-dd85-4870-97da-7c8c56eab46d None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:33.212564 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d314bb9-dd85-4870-97da-7c8c56eab46d None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:33.216447 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 185/358] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:33.216875 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d314bb9-dd85-4870-97da-7c8c56eab46d None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:33.245889 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48446e-d3e9-4168-b4f7-4f7569496a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:33.247393 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48446e-d3e9-4168-b4f7-4f7569496a0b None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:33.247741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48446e-d3e9-4168-b4f7-4f7569496a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:33.248091 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48446e-d3e9-4168-b4f7-4f7569496a0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:33.248912 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b48446e-d3e9-4168-b4f7-4f7569496a0b None admin] RBAC: Adding query filter params (group.id=87f7784d531244e9963fa10fbbf8ed8f, role.id=9265e1fd617b4c0b8fbf267885394060, scope.project.id=4367ec2ce611409a85975a2a505f1ff8) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:33.249672 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b48446e-d3e9-4168-b4f7-4f7569496a0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:33.250823 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b48446e-d3e9-4168-b4f7-4f7569496a0b None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:33.262821 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 174/359] 10.222.0.73 () {56 vars in 1376 bytes} [Mon Aug 29 19:59:33 2022] GET /identity/v3/role_assignments?group.id=87f7784d531244e9963fa10fbbf8ed8f&scope.project.id=4367ec2ce611409a85975a2a505f1ff8&role.id=9265e1fd617b4c0b8fbf267885394060 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:33.263535 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48446e-d3e9-4168-b4f7-4f7569496a0b None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:33.375258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:33.392666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d314bb9-dd85-4870-97da-7c8c56eab46d None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:33.439690 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48446e-d3e9-4168-b4f7-4f7569496a0b None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:33.442154 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d20f5-57c2-4a71-b32a-d6c49065fb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:33.442809 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d20f5-57c2-4a71-b32a-d6c49065fb1e None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:33.443098 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d20f5-57c2-4a71-b32a-d6c49065fb1e None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:33.443376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d20f5-57c2-4a71-b32a-d6c49065fb1e None admin] PATH_INFO: `/v3/users/76de9ca236aa4cce8ce8b64119c603a7` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:33.444573 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d20f5-57c2-4a71-b32a-d6c49065fb1e None admin] RBAC: Authorizing `identity:get_user(user_id=76de9ca236aa4cce8ce8b64119c603a7)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:33.446089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d20f5-57c2-4a71-b32a-d6c49065fb1e None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:33.448048 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 186/360] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 19:59:33 2022] GET /identity/v3/users/76de9ca236aa4cce8ce8b64119c603a7 => generated 293 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:33.449263 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd5fb9-b781-43c0-bd53-d92d4ae1bc3b None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:33.449829 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd5fb9-b781-43c0-bd53-d92d4ae1bc3b None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:33.450108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd5fb9-b781-43c0-bd53-d92d4ae1bc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:33.472546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-3fcd5fb9-b781-43c0-bd53-d92d4ae1bc3b None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:33.473812 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-3fcd5fb9-b781-43c0-bd53-d92d4ae1bc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:33.490068 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ad053-ce2b-4d6f-9e36-b198c59c351d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:33.490692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ad053-ce2b-4d6f-9e36-b198c59c351d None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:33.490976 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ad053-ce2b-4d6f-9e36-b198c59c351d None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:33.491250 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ad053-ce2b-4d6f-9e36-b198c59c351d None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:33.491930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ad053-ce2b-4d6f-9e36-b198c59c351d None admin] RBAC: Adding query filter params (name=member) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:33.492235 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ad053-ce2b-4d6f-9e36-b198c59c351d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:33.493628 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ad053-ce2b-4d6f-9e36-b198c59c351d None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:33.506308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 175/361] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:33.507573 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e0eda-1d97-480e-b964-937c67a9b988 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:33.507864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e0eda-1d97-480e-b964-937c67a9b988 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:33.508137 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e0eda-1d97-480e-b964-937c67a9b988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:33.513044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:33.527931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-8d5e0eda-1d97-480e-b964-937c67a9b988 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:33.529568 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-8d5e0eda-1d97-480e-b964-937c67a9b988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:33.565958 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 187/362] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:33.567425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd5fb9-b781-43c0-bd53-d92d4ae1bc3b None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:33.629754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 176/363] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:33.631469 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e0eda-1d97-480e-b964-937c67a9b988 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:33.726708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd5fb9-b781-43c0-bd53-d92d4ae1bc3b None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:33.727594 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:33.772425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47f279-0f7c-463f-9422-74b2144cc550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:33.773084 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47f279-0f7c-463f-9422-74b2144cc550 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:33.773373 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47f279-0f7c-463f-9422-74b2144cc550 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:33.773644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47f279-0f7c-463f-9422-74b2144cc550 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:33.774513 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47f279-0f7c-463f-9422-74b2144cc550 None admin] RBAC: Adding query filter params (role.id=c4b36aca8aba43d09071f52c763a80a8, user.id=76de9ca236aa4cce8ce8b64119c603a7, scope.system=all) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:33.774830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47f279-0f7c-463f-9422-74b2144cc550 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:33.776349 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47f279-0f7c-463f-9422-74b2144cc550 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:33.790853 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 188/364] 10.222.0.73 () {56 vars in 1308 bytes} [Mon Aug 29 19:59:33 2022] GET /identity/v3/role_assignments?user.id=76de9ca236aa4cce8ce8b64119c603a7&scope.system=all&role.id=c4b36aca8aba43d09071f52c763a80a8 => generated 510 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:33.803301 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e0eda-1d97-480e-b964-937c67a9b988 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:33.857772 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b200b-0e7f-4e8e-9368-4b4c097a72af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:33.858477 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b200b-0e7f-4e8e-9368-4b4c097a72af None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:33.858757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b200b-0e7f-4e8e-9368-4b4c097a72af None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:33.859030 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b200b-0e7f-4e8e-9368-4b4c097a72af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:33.899016 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47f279-0f7c-463f-9422-74b2144cc550 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:33.942463 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:33.949359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b200b-0e7f-4e8e-9368-4b4c097a72af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:33.950907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b200b-0e7f-4e8e-9368-4b4c097a72af None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:34.022084 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:34.076989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 177/365] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:34.078628 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b200b-0e7f-4e8e-9368-4b4c097a72af None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:34.128933 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47f279-0f7c-463f-9422-74b2144cc550 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:34.176620 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be452f-3ec7-49b3-8ee2-482653cdf4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:34.177427 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be452f-3ec7-49b3-8ee2-482653cdf4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:34.178204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be452f-3ec7-49b3-8ee2-482653cdf4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:34.178562 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be452f-3ec7-49b3-8ee2-482653cdf4e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:34.195015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be452f-3ec7-49b3-8ee2-482653cdf4e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:34.203465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b200b-0e7f-4e8e-9368-4b4c097a72af None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:34.204603 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-41be452f-3ec7-49b3-8ee2-482653cdf4e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:34.204603 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:34.204603 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:34.204603 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:34.204603 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:34.204603 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:34.204603 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:34.204603 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:34.204603 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:34.204603 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:34.204603 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:34.204603 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:34.204603 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:34.204603 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:34.204603 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:34.204603 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:34.205646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:34.205646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:34.205646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:34.205646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:34.205646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:34.205646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:34.205646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:34.205646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:34.205646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:34.205646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:34.205646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:34.205646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:34.205646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:34.205646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:34.205646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:34.205646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:34.206648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:34.206648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:34.206648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:34.206648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:34.206648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:34.206648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:34.206648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:34.206648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:34.206648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:34.206648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:34.206648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:34.206648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:34.209085 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 189/366] 10.222.0.73 () {56 vars in 1114 bytes} [Mon Aug 29 19:59:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:34.210094 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be452f-3ec7-49b3-8ee2-482653cdf4e7 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:34.253365 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4c6a1-68ef-44ce-828d-f89836a53553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:34.254296 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4c6a1-68ef-44ce-828d-f89836a53553 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:34.254745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4c6a1-68ef-44ce-828d-f89836a53553 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:34.255083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4c6a1-68ef-44ce-828d-f89836a53553 None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:34.256599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4c6a1-68ef-44ce-828d-f89836a53553 None admin] RBAC: Authorizing `identity:get_project(project_id=2e1ab10529d34aaabbf5bdd2cff84222)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:34.258791 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4c6a1-68ef-44ce-828d-f89836a53553 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:34.262900 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 178/367] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:34 2022] GET /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:34.264271 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4c6a1-68ef-44ce-828d-f89836a53553 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:34.299565 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:34.345662 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:34.374299 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be452f-3ec7-49b3-8ee2-482653cdf4e7 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:34.397219 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4c6a1-68ef-44ce-828d-f89836a53553 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:34.414033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af4232-288d-4af8-8e4a-86931514db7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:34.415163 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af4232-288d-4af8-8e4a-86931514db7b None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:34.415753 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af4232-288d-4af8-8e4a-86931514db7b None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:34.416226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af4232-288d-4af8-8e4a-86931514db7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:34.430457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af4232-288d-4af8-8e4a-86931514db7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:34.436826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-c7af4232-288d-4af8-8e4a-86931514db7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:59:34.436826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:34.436826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:34.436826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:34.436826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:34.436826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:34.436826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:34.436826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:34.436826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:34.436826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:34.436826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:34.436826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:34.436826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:34.436826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:34.436826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:34.436826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:34.438033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:34.438033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:34.438033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:34.438033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:34.438033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:34.438033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:34.438033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:34.438033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:34.438033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:34.438033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:34.438033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:34.438033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:34.438033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:34.438033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:34.438033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:34.438033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:34.438880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:34.438880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:34.438880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:34.438880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:34.438880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:34.438880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:34.438880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:34.438880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:34.438880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:34.438880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:34.438880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:59:34.438880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:34.440033 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27151750-3d56-4123-9e18-c4677da8ec42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:34.440459 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27151750-3d56-4123-9e18-c4677da8ec42 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:34.440731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27151750-3d56-4123-9e18-c4677da8ec42 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:34.441001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27151750-3d56-4123-9e18-c4677da8ec42 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:34.441665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27151750-3d56-4123-9e18-c4677da8ec42 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:34.442082 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27151750-3d56-4123-9e18-c4677da8ec42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:34.442598 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 190/368] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 19:59:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:34.443806 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af4232-288d-4af8-8e4a-86931514db7b None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:34.444083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27151750-3d56-4123-9e18-c4677da8ec42 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:34.455815 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 179/369] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 19:59:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:34.456679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27151750-3d56-4123-9e18-c4677da8ec42 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:34.552333 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:34.570961 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af4232-288d-4af8-8e4a-86931514db7b None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:34.578288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27151750-3d56-4123-9e18-c4677da8ec42 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:34.619221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6c9aa-6c95-4bec-866e-9bfd8bc9815e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:34.619864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6c9aa-6c95-4bec-866e-9bfd8bc9815e None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:34.620138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6c9aa-6c95-4bec-866e-9bfd8bc9815e None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:34.620407 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6c9aa-6c95-4bec-866e-9bfd8bc9815e None admin] PATH_INFO: `/v3/groups/87f7784d531244e9963fa10fbbf8ed8f` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:34.623104 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba6c9aa-6c95-4bec-866e-9bfd8bc9815e None admin] RBAC: Authorizing `identity:get_group(group_id=87f7784d531244e9963fa10fbbf8ed8f)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:34.624743 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba6c9aa-6c95-4bec-866e-9bfd8bc9815e None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:34.627291 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 191/370] 10.222.0.73 () {56 vars in 1180 bytes} [Mon Aug 29 19:59:34 2022] GET /identity/v3/groups/87f7784d531244e9963fa10fbbf8ed8f => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:34.627964 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6c9aa-6c95-4bec-866e-9bfd8bc9815e None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:34.631620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84782b-060b-4a4d-916c-001419662380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:34.633909 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84782b-060b-4a4d-916c-001419662380 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:34.634405 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84782b-060b-4a4d-916c-001419662380 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:34.634876 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84782b-060b-4a4d-916c-001419662380 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:34.635821 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84782b-060b-4a4d-916c-001419662380 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:34.636330 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84782b-060b-4a4d-916c-001419662380 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:34.640507 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84782b-060b-4a4d-916c-001419662380 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:34.641546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:34.655598 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 180/371] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:34.657262 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84782b-060b-4a4d-916c-001419662380 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:34.770409 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6c9aa-6c95-4bec-866e-9bfd8bc9815e None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:34.793764 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84782b-060b-4a4d-916c-001419662380 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:34.811102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddae79-6956-44db-b15c-756e99c196f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:34.811727 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddae79-6956-44db-b15c-756e99c196f7 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:34.812024 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddae79-6956-44db-b15c-756e99c196f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:34.812282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddae79-6956-44db-b15c-756e99c196f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:34.835208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc7976-bf8e-47c3-848a-d4118c3bf99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:34.836204 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc7976-bf8e-47c3-848a-d4118c3bf99e None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:34.836708 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc7976-bf8e-47c3-848a-d4118c3bf99e None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:34.837179 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc7976-bf8e-47c3-848a-d4118c3bf99e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:34.838154 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc7976-bf8e-47c3-848a-d4118c3bf99e None admin] RBAC: Adding query filter params (group.id=87f7784d531244e9963fa10fbbf8ed8f, role.id=9265e1fd617b4c0b8fbf267885394060, scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:34.838683 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc7976-bf8e-47c3-848a-d4118c3bf99e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:34.840379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc7976-bf8e-47c3-848a-d4118c3bf99e None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:34.846969 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 181/372] 10.222.0.73 () {56 vars in 1376 bytes} [Mon Aug 29 19:59:34 2022] GET /identity/v3/role_assignments?group.id=87f7784d531244e9963fa10fbbf8ed8f&scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222&role.id=9265e1fd617b4c0b8fbf267885394060 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:34.848136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc7976-bf8e-47c3-848a-d4118c3bf99e None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:34.869384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:34.895652 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ddae79-6956-44db-b15c-756e99c196f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:34.897171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ddae79-6956-44db-b15c-756e99c196f7 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:34.910116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:35.004216 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc7976-bf8e-47c3-848a-d4118c3bf99e None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:35.042804 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94484fcf-13b7-4fa8-9d3b-abff1bef20b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:35.043653 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94484fcf-13b7-4fa8-9d3b-abff1bef20b6 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:35.044142 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94484fcf-13b7-4fa8-9d3b-abff1bef20b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:35.044505 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94484fcf-13b7-4fa8-9d3b-abff1bef20b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:35.045558 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 192/373] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:35.052026 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddae79-6956-44db-b15c-756e99c196f7 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:35.124628 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94484fcf-13b7-4fa8-9d3b-abff1bef20b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:35.126227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94484fcf-13b7-4fa8-9d3b-abff1bef20b6 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:35.134926 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:35.251488 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddae79-6956-44db-b15c-756e99c196f7 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:35.269292 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:35.273116 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 182/374] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:35.281618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94484fcf-13b7-4fa8-9d3b-abff1bef20b6 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:35.303170 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eaee9f-2ca9-418e-a8ec-b3cc3156088b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:35.303738 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eaee9f-2ca9-418e-a8ec-b3cc3156088b None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:35.304033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eaee9f-2ca9-418e-a8ec-b3cc3156088b None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:35.304302 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eaee9f-2ca9-418e-a8ec-b3cc3156088b None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:35.307661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eaee9f-2ca9-418e-a8ec-b3cc3156088b None admin] RBAC: Authorizing `identity:get_project(project_id=2e1ab10529d34aaabbf5bdd2cff84222)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:35.312008 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eaee9f-2ca9-418e-a8ec-b3cc3156088b None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:35.313785 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 193/375] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:35 2022] GET /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222 => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:35.320661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eaee9f-2ca9-418e-a8ec-b3cc3156088b None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:35.476792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94484fcf-13b7-4fa8-9d3b-abff1bef20b6 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:35.485064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:35.486336 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eaee9f-2ca9-418e-a8ec-b3cc3156088b None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:35.525066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754336ce-5ae3-44ae-b9b1-28260026c819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:35.525772 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754336ce-5ae3-44ae-b9b1-28260026c819 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:35.526112 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754336ce-5ae3-44ae-b9b1-28260026c819 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:35.526394 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754336ce-5ae3-44ae-b9b1-28260026c819 None admin] PATH_INFO: `/v3/projects/ee34b4bd3d8043258a1a5d3588fa8f9c` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:35.527695 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754336ce-5ae3-44ae-b9b1-28260026c819 None admin] RBAC: Authorizing `identity:get_project(project_id=ee34b4bd3d8043258a1a5d3588fa8f9c)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:35.529360 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754336ce-5ae3-44ae-b9b1-28260026c819 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:35.535744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 183/376] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:35 2022] GET /identity/v3/projects/ee34b4bd3d8043258a1a5d3588fa8f9c => generated 344 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:35.536617 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754336ce-5ae3-44ae-b9b1-28260026c819 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:35.548676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56d4df-37e7-4d49-b4bc-48640efc67ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:35.549319 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56d4df-37e7-4d49-b4bc-48640efc67ec None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:35.549595 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56d4df-37e7-4d49-b4bc-48640efc67ec None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:35.549896 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56d4df-37e7-4d49-b4bc-48640efc67ec None admin] PATH_INFO: `/v3/groups/87f7784d531244e9963fa10fbbf8ed8f` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:35.553386 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56d4df-37e7-4d49-b4bc-48640efc67ec None admin] RBAC: Authorizing `identity:get_group(group_id=87f7784d531244e9963fa10fbbf8ed8f)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:35.554826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56d4df-37e7-4d49-b4bc-48640efc67ec None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:35.562637 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 194/377] 10.222.0.73 () {56 vars in 1180 bytes} [Mon Aug 29 19:59:35 2022] GET /identity/v3/groups/87f7784d531244e9963fa10fbbf8ed8f => generated 226 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:35.588597 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56d4df-37e7-4d49-b4bc-48640efc67ec None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:35.651509 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:35.726648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56d4df-37e7-4d49-b4bc-48640efc67ec None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:35.766914 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c26fd-3d8e-4b39-b03c-3310fe9a6540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:35.767525 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c26fd-3d8e-4b39-b03c-3310fe9a6540 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:35.767799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c26fd-3d8e-4b39-b03c-3310fe9a6540 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:35.768090 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c26fd-3d8e-4b39-b03c-3310fe9a6540 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:35.773733 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c26fd-3d8e-4b39-b03c-3310fe9a6540 None admin] RBAC: Adding query filter params (group.id=87f7784d531244e9963fa10fbbf8ed8f, role.id=b9184599d95b4950beca34304cdd2eb8, scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:35.773733 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c26fd-3d8e-4b39-b03c-3310fe9a6540 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:35.773733 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c26fd-3d8e-4b39-b03c-3310fe9a6540 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:35.781333 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 195/378] 10.222.0.73 () {56 vars in 1376 bytes} [Mon Aug 29 19:59:35 2022] GET /identity/v3/role_assignments?group.id=87f7784d531244e9963fa10fbbf8ed8f&scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222&role.id=b9184599d95b4950beca34304cdd2eb8 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:35.794683 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754336ce-5ae3-44ae-b9b1-28260026c819 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:35.845439 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d00519-a726-4af1-b051-f82fd0712077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:35.846124 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d00519-a726-4af1-b051-f82fd0712077 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:35.846399 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d00519-a726-4af1-b051-f82fd0712077 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:35.846667 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d00519-a726-4af1-b051-f82fd0712077 None admin] PATH_INFO: `/v3/groups/dff53ed4f9a94eee82f8d8ef3b0c098d` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:35.858811 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d00519-a726-4af1-b051-f82fd0712077 None admin] RBAC: Authorizing `identity:get_group(group_id=dff53ed4f9a94eee82f8d8ef3b0c098d)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:35.860745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d00519-a726-4af1-b051-f82fd0712077 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:35.863032 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 184/379] 10.222.0.73 () {56 vars in 1180 bytes} [Mon Aug 29 19:59:35 2022] GET /identity/v3/groups/dff53ed4f9a94eee82f8d8ef3b0c098d => generated 229 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:35.868683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c26fd-3d8e-4b39-b03c-3310fe9a6540 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:35.927856 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3f731-abf7-4ae9-a212-3db09f44e051 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:35.928179 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3f731-abf7-4ae9-a212-3db09f44e051 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:35.928395 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3f731-abf7-4ae9-a212-3db09f44e051 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:35.930511 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 185/380] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:35.932162 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74606d62-b149-45b4-b712-06dae53d759f None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:35.932442 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74606d62-b149-45b4-b712-06dae53d759f None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:35.932779 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74606d62-b149-45b4-b712-06dae53d759f None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:35.934262 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 186/381] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:59:35.937927 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace9e49-c7fd-47af-b4f7-2c1f7077f1f4 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:35.938231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace9e49-c7fd-47af-b4f7-2c1f7077f1f4 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:35.938592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace9e49-c7fd-47af-b4f7-2c1f7077f1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:35.963425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-eace9e49-c7fd-47af-b4f7-2c1f7077f1f4 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:35.965230 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-eace9e49-c7fd-47af-b4f7-2c1f7077f1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:36.006377 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c26fd-3d8e-4b39-b03c-3310fe9a6540 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:36.014745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:36.018765 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:36.054212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398021e-a944-4d3d-833e-4a1b934c65ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:36.054829 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398021e-a944-4d3d-833e-4a1b934c65ac None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:36.055108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398021e-a944-4d3d-833e-4a1b934c65ac None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:36.055380 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398021e-a944-4d3d-833e-4a1b934c65ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:36.056233 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398021e-a944-4d3d-833e-4a1b934c65ac None admin] RBAC: Adding query filter params (group.id=dff53ed4f9a94eee82f8d8ef3b0c098d, role.id=296ee2c194824485af71618619f82e09, scope.project.id=ee34b4bd3d8043258a1a5d3588fa8f9c) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:36.056544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398021e-a944-4d3d-833e-4a1b934c65ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:36.059353 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398021e-a944-4d3d-833e-4a1b934c65ac None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:36.066238 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 196/382] 10.222.0.73 () {56 vars in 1376 bytes} [Mon Aug 29 19:59:35 2022] GET /identity/v3/role_assignments?group.id=dff53ed4f9a94eee82f8d8ef3b0c098d&scope.project.id=ee34b4bd3d8043258a1a5d3588fa8f9c&role.id=296ee2c194824485af71618619f82e09 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:36.067570 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca640a2-0b71-469d-97eb-78fd10b06296 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:36.067860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca640a2-0b71-469d-97eb-78fd10b06296 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:36.068136 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca640a2-0b71-469d-97eb-78fd10b06296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:36.075175 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 187/383] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:59:36.103414 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-9ca640a2-0b71-469d-97eb-78fd10b06296 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:36.104929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-9ca640a2-0b71-469d-97eb-78fd10b06296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:36.210218 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 197/384] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:36.413537 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace9e49-c7fd-47af-b4f7-2c1f7077f1f4 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:36.528680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace9e49-c7fd-47af-b4f7-2c1f7077f1f4 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:36.569051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacf1ec-7e68-4622-9811-08587ac3e5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:36.569994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacf1ec-7e68-4622-9811-08587ac3e5a2 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:36.570409 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacf1ec-7e68-4622-9811-08587ac3e5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:36.570810 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacf1ec-7e68-4622-9811-08587ac3e5a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:36.583965 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edacf1ec-7e68-4622-9811-08587ac3e5a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:36.589247 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-edacf1ec-7e68-4622-9811-08587ac3e5a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:36.589247 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:36.589247 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:36.589247 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:36.589247 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:36.589247 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:36.589247 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:36.589247 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:36.589247 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:36.589247 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:36.589247 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:36.589247 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:36.589247 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:36.589247 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:36.589247 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:36.589247 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:36.590850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:36.590850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:36.590850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:36.590850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:36.590850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:36.590850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:36.590850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:36.590850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:36.590850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:36.590850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:36.590850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:36.590850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:36.590850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:36.590850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:36.590850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:36.590850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:36.591709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:36.591709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:36.591709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:36.591709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:36.591709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:36.591709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:36.591709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:36.591709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:36.591709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:36.591709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:36.591709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:36.591709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:36.597808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 188/385] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:36.599874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca640a2-0b71-469d-97eb-78fd10b06296 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:36.724892 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca640a2-0b71-469d-97eb-78fd10b06296 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:36.726643 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:36.744731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacf1ec-7e68-4622-9811-08587ac3e5a2 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:36.788399 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:36.791438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d879afc-3020-45b6-ad32-c675c9e91e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:36.792066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d879afc-3020-45b6-ad32-c675c9e91e60 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:36.792343 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d879afc-3020-45b6-ad32-c675c9e91e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:36.792613 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d879afc-3020-45b6-ad32-c675c9e91e60 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:36.793352 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d879afc-3020-45b6-ad32-c675c9e91e60 None admin] RBAC: Adding query filter params (name=member) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:36.793655 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d879afc-3020-45b6-ad32-c675c9e91e60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:36.795068 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d879afc-3020-45b6-ad32-c675c9e91e60 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:36.813228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 198/386] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:36.818723 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d879afc-3020-45b6-ad32-c675c9e91e60 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:36.889659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacf1ec-7e68-4622-9811-08587ac3e5a2 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:36.959052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a0246-5c79-4193-a58a-a709dc572b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:36.959690 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a0246-5c79-4193-a58a-a709dc572b6c None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:36.959962 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a0246-5c79-4193-a58a-a709dc572b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:36.960224 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a0246-5c79-4193-a58a-a709dc572b6c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:36.980176 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a0246-5c79-4193-a58a-a709dc572b6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:36.986842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-a64a0246-5c79-4193-a58a-a709dc572b6c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:36.986842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:36.986842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:36.986842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:36.986842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:36.986842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:36.986842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:36.986842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:36.986842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:36.986842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:36.986842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:36.986842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:36.986842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:36.986842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:36.986842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:36.986842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:36.987731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:36.987731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:36.987731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:36.987731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:36.987731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:36.987731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:36.987731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:36.987731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:36.987731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:36.987731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:36.987731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:36.987731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:36.987731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:36.987731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:36.987731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:36.987731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:36.988502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:36.988502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:36.988502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:36.988502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:36.988502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:36.988502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:36.988502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:36.988502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:36.988502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:36.988502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:36.988502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:36.988502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:37.001790 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 189/387] 10.222.0.73 () {56 vars in 1114 bytes} [Mon Aug 29 19:59:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 19:59:37.004297 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a0246-5c79-4193-a58a-a709dc572b6c None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:37.015629 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d879afc-3020-45b6-ad32-c675c9e91e60 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:37.075936 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0269608-e631-47df-89e6-312c44f8f896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:37.076586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0269608-e631-47df-89e6-312c44f8f896 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:37.076864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0269608-e631-47df-89e6-312c44f8f896 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:37.077139 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0269608-e631-47df-89e6-312c44f8f896 None admin] PATH_INFO: `/v3/groups/87f7784d531244e9963fa10fbbf8ed8f` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:37.078990 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0269608-e631-47df-89e6-312c44f8f896 None admin] RBAC: Authorizing `identity:get_group(group_id=87f7784d531244e9963fa10fbbf8ed8f)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:37.080462 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0269608-e631-47df-89e6-312c44f8f896 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:37.084747 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 199/388] 10.222.0.73 () {56 vars in 1180 bytes} [Mon Aug 29 19:59:36 2022] GET /identity/v3/groups/87f7784d531244e9963fa10fbbf8ed8f => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:37.092750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0269608-e631-47df-89e6-312c44f8f896 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:37.146101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:37.241818 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:37.245632 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0269608-e631-47df-89e6-312c44f8f896 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:37.280046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a0246-5c79-4193-a58a-a709dc572b6c None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:37.292291 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbc15a-ea91-4b1a-aec8-48783f572945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:37.292922 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbc15a-ea91-4b1a-aec8-48783f572945 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:37.293872 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbc15a-ea91-4b1a-aec8-48783f572945 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:37.294865 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbc15a-ea91-4b1a-aec8-48783f572945 None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:37.296042 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dbc15a-ea91-4b1a-aec8-48783f572945 None admin] RBAC: Authorizing `identity:get_project(project_id=4367ec2ce611409a85975a2a505f1ff8)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:37.297607 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dbc15a-ea91-4b1a-aec8-48783f572945 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:37.300126 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 200/389] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:37 2022] GET /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:37.306397 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbc15a-ea91-4b1a-aec8-48783f572945 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:37.367593 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878c4d8-e7b2-44bf-b42d-2a1bea68290a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:37.368223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878c4d8-e7b2-44bf-b42d-2a1bea68290a None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:37.368508 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878c4d8-e7b2-44bf-b42d-2a1bea68290a None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:37.368778 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878c4d8-e7b2-44bf-b42d-2a1bea68290a None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:37.369452 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878c4d8-e7b2-44bf-b42d-2a1bea68290a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:37.369776 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878c4d8-e7b2-44bf-b42d-2a1bea68290a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:37.374905 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878c4d8-e7b2-44bf-b42d-2a1bea68290a None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:37.400881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 190/390] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 19:59:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:37.410436 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878c4d8-e7b2-44bf-b42d-2a1bea68290a None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:37.446443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbc15a-ea91-4b1a-aec8-48783f572945 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:37.454766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:37.502284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f541cdd-d920-48d2-afd4-aea642933a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:37.503298 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f541cdd-d920-48d2-afd4-aea642933a6c None admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:37.503777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f541cdd-d920-48d2-afd4-aea642933a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:37.504393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f541cdd-d920-48d2-afd4-aea642933a6c None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8/groups/87f7784d531244e9963fa10fbbf8ed8f/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:37.507450 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f541cdd-d920-48d2-afd4-aea642933a6c None admin] RBAC: Authorizing `identity:create_grant(project_id=4367ec2ce611409a85975a2a505f1ff8, group_id=87f7784d531244e9963fa10fbbf8ed8f, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:37.510293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f541cdd-d920-48d2-afd4-aea642933a6c None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:37.531766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 201/391] 10.222.0.73 () {58 vars in 1442 bytes} [Mon Aug 29 19:59:37 2022] PUT /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8/groups/87f7784d531244e9963fa10fbbf8ed8f/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:59:37.534220 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e44b37-1d30-4c70-bf6f-09c3331e4d20 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:37.534858 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e44b37-1d30-4c70-bf6f-09c3331e4d20 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:37.535376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e44b37-1d30-4c70-bf6f-09c3331e4d20 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:37.537799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 202/392] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:37.544559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece884d7-e9b0-4ac8-8897-d29271af0e24 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:37.544892 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece884d7-e9b0-4ac8-8897-d29271af0e24 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:37.545105 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece884d7-e9b0-4ac8-8897-d29271af0e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:37.561353 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878c4d8-e7b2-44bf-b42d-2a1bea68290a None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:37.570844 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-ece884d7-e9b0-4ac8-8897-d29271af0e24 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:37.572624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ece884d7-e9b0-4ac8-8897-d29271af0e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:37.602625 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0b91c-3109-4499-8c5a-96c515e57de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:37.603279 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0b91c-3109-4499-8c5a-96c515e57de8 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:37.603559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0b91c-3109-4499-8c5a-96c515e57de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:37.603829 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0b91c-3109-4499-8c5a-96c515e57de8 None admin] PATH_INFO: `/v3/groups/87f7784d531244e9963fa10fbbf8ed8f` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:37.605265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0b91c-3109-4499-8c5a-96c515e57de8 None admin] RBAC: Authorizing `identity:get_group(group_id=87f7784d531244e9963fa10fbbf8ed8f)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:37.607717 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0b91c-3109-4499-8c5a-96c515e57de8 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:37.613789 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 191/393] 10.222.0.73 () {56 vars in 1180 bytes} [Mon Aug 29 19:59:37 2022] GET /identity/v3/groups/87f7784d531244e9963fa10fbbf8ed8f => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:37.616356 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0b91c-3109-4499-8c5a-96c515e57de8 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:37.646760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:37.682278 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 203/394] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:37.756291 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0b91c-3109-4499-8c5a-96c515e57de8 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:37.795299 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1507a9a-cc27-44e9-9208-c2bc5a07cb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:37.795953 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1507a9a-cc27-44e9-9208-c2bc5a07cb56 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:37.796232 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1507a9a-cc27-44e9-9208-c2bc5a07cb56 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:37.796558 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1507a9a-cc27-44e9-9208-c2bc5a07cb56 None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:37.798106 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1507a9a-cc27-44e9-9208-c2bc5a07cb56 None admin] RBAC: Authorizing `identity:get_project(project_id=4367ec2ce611409a85975a2a505f1ff8)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:37.799639 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1507a9a-cc27-44e9-9208-c2bc5a07cb56 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:37.801638 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 192/395] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:37 2022] GET /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:37.807482 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece884d7-e9b0-4ac8-8897-d29271af0e24 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:37.919630 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:37.931620 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece884d7-e9b0-4ac8-8897-d29271af0e24 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:37.973224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87695730-530f-4384-a0d1-38093cd7a037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:37.973934 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87695730-530f-4384-a0d1-38093cd7a037 None admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:37.974213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87695730-530f-4384-a0d1-38093cd7a037 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:37.974486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87695730-530f-4384-a0d1-38093cd7a037 None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8/groups/87f7784d531244e9963fa10fbbf8ed8f/roles/b9184599d95b4950beca34304cdd2eb8` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:37.976801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87695730-530f-4384-a0d1-38093cd7a037 None admin] RBAC: Authorizing `identity:create_grant(project_id=4367ec2ce611409a85975a2a505f1ff8, group_id=87f7784d531244e9963fa10fbbf8ed8f, role_id=b9184599d95b4950beca34304cdd2eb8)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:37.978378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87695730-530f-4384-a0d1-38093cd7a037 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:37.993213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 204/396] 10.222.0.73 () {58 vars in 1442 bytes} [Mon Aug 29 19:59:37 2022] PUT /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8/groups/87f7784d531244e9963fa10fbbf8ed8f/roles/b9184599d95b4950beca34304cdd2eb8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:59:38.027429 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1507a9a-cc27-44e9-9208-c2bc5a07cb56 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:38.169743 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:38.190445 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1507a9a-cc27-44e9-9208-c2bc5a07cb56 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:38.227082 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8c3fd-3efd-40d1-a991-cfccf1d3d176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:38.227722 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8c3fd-3efd-40d1-a991-cfccf1d3d176 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:38.228002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8c3fd-3efd-40d1-a991-cfccf1d3d176 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:38.228272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8c3fd-3efd-40d1-a991-cfccf1d3d176 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:38.240386 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8c3fd-3efd-40d1-a991-cfccf1d3d176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:38.247786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-2ba8c3fd-3efd-40d1-a991-cfccf1d3d176 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:38.247786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:38.247786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:38.247786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:38.247786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:38.247786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:38.247786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:38.247786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:38.247786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:38.247786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:38.247786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:38.247786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:38.247786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:38.247786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:38.247786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:38.247786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:38.249078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:38.249078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:38.249078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:38.249078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:38.249078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:38.249078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:38.249078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:38.249078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:38.249078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:38.249078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:38.249078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:38.249078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:38.249078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:38.249078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:38.249078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:38.249078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:38.250221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:38.250221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:38.250221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:38.250221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:38.250221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:38.250221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:38.250221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:38.250221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:38.250221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:38.250221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:38.250221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:38.250221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:38.250221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 193/397] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:38.257311 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87695730-530f-4384-a0d1-38093cd7a037 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:38.302322 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afdf32-07f1-4eda-b0f8-a3ec35692cee None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:38.302601 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afdf32-07f1-4eda-b0f8-a3ec35692cee None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:38.302768 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afdf32-07f1-4eda-b0f8-a3ec35692cee None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:38.304478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 194/398] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:59:38.319791 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcf1b9-cb27-4b3c-ae00-f4e4d49d3c91 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:38.320104 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcf1b9-cb27-4b3c-ae00-f4e4d49d3c91 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:38.320336 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcf1b9-cb27-4b3c-ae00-f4e4d49d3c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:38.353567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-4fbcf1b9-cb27-4b3c-ae00-f4e4d49d3c91 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:38.354822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-4fbcf1b9-cb27-4b3c-ae00-f4e4d49d3c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:38.385151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87695730-530f-4384-a0d1-38093cd7a037 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:38.422886 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770954e2-5797-46cd-8ce3-0db7f4ba8373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:38.423506 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770954e2-5797-46cd-8ce3-0db7f4ba8373 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:38.423787 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770954e2-5797-46cd-8ce3-0db7f4ba8373 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:38.424058 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770954e2-5797-46cd-8ce3-0db7f4ba8373 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:38.424729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770954e2-5797-46cd-8ce3-0db7f4ba8373 None admin] RBAC: Adding query filter params (name=member) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:38.425028 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770954e2-5797-46cd-8ce3-0db7f4ba8373 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:38.426525 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770954e2-5797-46cd-8ce3-0db7f4ba8373 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:38.436397 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 205/399] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:38.445919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770954e2-5797-46cd-8ce3-0db7f4ba8373 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:38.486080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:38.522727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 195/400] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:38.591995 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770954e2-5797-46cd-8ce3-0db7f4ba8373 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:38.628583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8b09d-cffd-45da-a134-1a111da82933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:38.629392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8b09d-cffd-45da-a134-1a111da82933 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:38.629722 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8b09d-cffd-45da-a134-1a111da82933 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:38.630007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8b09d-cffd-45da-a134-1a111da82933 None admin] PATH_INFO: `/v3/groups/87f7784d531244e9963fa10fbbf8ed8f` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:38.631147 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8b09d-cffd-45da-a134-1a111da82933 None admin] RBAC: Authorizing `identity:get_group(group_id=87f7784d531244e9963fa10fbbf8ed8f)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:38.632587 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8b09d-cffd-45da-a134-1a111da82933 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:38.634503 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 206/401] 10.222.0.73 () {56 vars in 1180 bytes} [Mon Aug 29 19:59:38 2022] GET /identity/v3/groups/87f7784d531244e9963fa10fbbf8ed8f => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:38.642899 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbcf1b9-cb27-4b3c-ae00-f4e4d49d3c91 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:38.754348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:38.781068 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbcf1b9-cb27-4b3c-ae00-f4e4d49d3c91 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:38.825919 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57265a-8d6a-4b39-8a25-5d1190cdef43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:38.826567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57265a-8d6a-4b39-8a25-5d1190cdef43 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:38.826842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57265a-8d6a-4b39-8a25-5d1190cdef43 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:38.827112 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57265a-8d6a-4b39-8a25-5d1190cdef43 None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:38.828266 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f57265a-8d6a-4b39-8a25-5d1190cdef43 None admin] RBAC: Authorizing `identity:get_project(project_id=2e1ab10529d34aaabbf5bdd2cff84222)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:38.829723 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f57265a-8d6a-4b39-8a25-5d1190cdef43 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:38.831368 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 196/402] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:38 2022] GET /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:38.837380 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8b09d-cffd-45da-a134-1a111da82933 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:38.912724 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fd0b6-e4f3-4115-9ef5-19ab96647b9f None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:38.912909 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fd0b6-e4f3-4115-9ef5-19ab96647b9f None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:38.913172 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fd0b6-e4f3-4115-9ef5-19ab96647b9f None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:38.915891 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 197/403] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:38.920388 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa4e20-fd9f-43f2-8a4f-3d4a1bcd5896 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:38.920664 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa4e20-fd9f-43f2-8a4f-3d4a1bcd5896 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:38.920963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa4e20-fd9f-43f2-8a4f-3d4a1bcd5896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:38.955218 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:38.975515 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8b09d-cffd-45da-a134-1a111da82933 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:38.976695 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-84fa4e20-fd9f-43f2-8a4f-3d4a1bcd5896 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:38.978085 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-84fa4e20-fd9f-43f2-8a4f-3d4a1bcd5896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:39.013844 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f2c78-0408-4d65-8a48-6e2c8aab72e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:39.014568 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f2c78-0408-4d65-8a48-6e2c8aab72e9 None admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:39.014850 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f2c78-0408-4d65-8a48-6e2c8aab72e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:39.015122 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f2c78-0408-4d65-8a48-6e2c8aab72e9 None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222/groups/87f7784d531244e9963fa10fbbf8ed8f/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:39.017489 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879f2c78-0408-4d65-8a48-6e2c8aab72e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e1ab10529d34aaabbf5bdd2cff84222, group_id=87f7784d531244e9963fa10fbbf8ed8f, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:39.018523 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879f2c78-0408-4d65-8a48-6e2c8aab72e9 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:39.041808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 207/404] 10.222.0.73 () {58 vars in 1442 bytes} [Mon Aug 29 19:59:38 2022] PUT /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222/groups/87f7784d531244e9963fa10fbbf8ed8f/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:59:39.041808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f2c78-0408-4d65-8a48-6e2c8aab72e9 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:39.072105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 198/405] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:59:39.161664 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f2c78-0408-4d65-8a48-6e2c8aab72e9 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:39.199301 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d31e0-f9e7-4c32-8f06-2502cbcbfc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:39.200690 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d31e0-f9e7-4c32-8f06-2502cbcbfc4c None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:39.201069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d31e0-f9e7-4c32-8f06-2502cbcbfc4c None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:39.201339 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d31e0-f9e7-4c32-8f06-2502cbcbfc4c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:39.209984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:39.218862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d31e0-f9e7-4c32-8f06-2502cbcbfc4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:39.226637 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-9b4d31e0-f9e7-4c32-8f06-2502cbcbfc4c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:39.226637 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:39.226637 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:39.226637 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:39.226637 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:39.226637 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:39.226637 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:39.226637 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:39.226637 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:39.226637 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:39.226637 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:39.226637 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:39.226637 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:39.226637 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:39.226637 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:39.226637 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:39.227786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:39.227786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:39.227786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:39.227786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:39.227786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:39.227786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:39.227786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:39.227786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:39.227786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:39.227786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:39.227786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:39.227786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:39.227786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:39.227786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:39.227786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:39.227786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:39.228752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:39.228752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:39.228752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:39.228752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:39.228752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:39.228752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:39.228752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:39.228752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:39.228752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:39.228752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:39.228752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:39.228752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:39.228752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 208/406] 10.222.0.73 () {56 vars in 1114 bytes} [Mon Aug 29 19:59:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:39.233875 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa4e20-fd9f-43f2-8a4f-3d4a1bcd5896 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:39.363522 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:39.399379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa4e20-fd9f-43f2-8a4f-3d4a1bcd5896 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:39.437204 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0193cba-68d9-453f-b7cb-7705be4aa801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:39.437845 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0193cba-68d9-453f-b7cb-7705be4aa801 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:39.438121 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0193cba-68d9-453f-b7cb-7705be4aa801 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:39.438389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0193cba-68d9-453f-b7cb-7705be4aa801 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:39.439046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0193cba-68d9-453f-b7cb-7705be4aa801 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:39.439344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0193cba-68d9-453f-b7cb-7705be4aa801 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:39.440699 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0193cba-68d9-453f-b7cb-7705be4aa801 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:39.450250 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 199/407] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 19:59:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:39.454432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d31e0-f9e7-4c32-8f06-2502cbcbfc4c None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:39.455509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0193cba-68d9-453f-b7cb-7705be4aa801 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:39.580521 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0193cba-68d9-453f-b7cb-7705be4aa801 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:39.647906 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d31e0-f9e7-4c32-8f06-2502cbcbfc4c None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:39.650354 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa4623-60de-4ddb-9e28-40884b58599c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:39.651921 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa4623-60de-4ddb-9e28-40884b58599c None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:39.652214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa4623-60de-4ddb-9e28-40884b58599c None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:39.652491 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa4623-60de-4ddb-9e28-40884b58599c None admin] PATH_INFO: `/v3/groups/87f7784d531244e9963fa10fbbf8ed8f` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:39.654315 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa4623-60de-4ddb-9e28-40884b58599c None admin] RBAC: Authorizing `identity:get_group(group_id=87f7784d531244e9963fa10fbbf8ed8f)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:39.655787 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa4623-60de-4ddb-9e28-40884b58599c None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:39.657719 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 200/408] 10.222.0.73 () {56 vars in 1180 bytes} [Mon Aug 29 19:59:39 2022] GET /identity/v3/groups/87f7784d531244e9963fa10fbbf8ed8f => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:39.666381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa4623-60de-4ddb-9e28-40884b58599c None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:39.673733 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:39.728784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6f964-a8ff-4809-a03f-f3fc1d13c0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:39.729345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6f964-a8ff-4809-a03f-f3fc1d13c0bc None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:39.729623 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6f964-a8ff-4809-a03f-f3fc1d13c0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:39.729919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6f964-a8ff-4809-a03f-f3fc1d13c0bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:39.751553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af6f964-a8ff-4809-a03f-f3fc1d13c0bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:39.757485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-0af6f964-a8ff-4809-a03f-f3fc1d13c0bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:59:39.757485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:39.757485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:39.757485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:39.757485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:39.757485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:39.757485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:39.757485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:39.757485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:39.757485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:39.757485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:39.757485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:39.757485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:39.757485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:39.757485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:39.757485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:39.758404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:39.758404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:39.758404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:39.758404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:39.758404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:39.758404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:39.758404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:39.758404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:39.758404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:39.758404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:39.758404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:39.758404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:39.758404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:39.758404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:39.758404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:39.758404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:39.759205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:39.759205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:39.759205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:39.759205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:39.759205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:39.759205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:39.759205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:39.759205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:39.759205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:39.759205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:39.759205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:59:39.759205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:39.759205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 209/409] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 19:59:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:39.769548 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6f964-a8ff-4809-a03f-f3fc1d13c0bc None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:39.862940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa4623-60de-4ddb-9e28-40884b58599c None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:39.873063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:39.934400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc2d29-fb27-42df-9f47-b7dec1d75375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:39.935046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc2d29-fb27-42df-9f47-b7dec1d75375 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:39.935323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc2d29-fb27-42df-9f47-b7dec1d75375 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:39.935596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc2d29-fb27-42df-9f47-b7dec1d75375 None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:39.937131 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc2d29-fb27-42df-9f47-b7dec1d75375 None admin] RBAC: Authorizing `identity:get_project(project_id=2e1ab10529d34aaabbf5bdd2cff84222)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:39.938956 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc2d29-fb27-42df-9f47-b7dec1d75375 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:39.941234 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 201/410] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:39 2022] GET /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:39.947667 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc2d29-fb27-42df-9f47-b7dec1d75375 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:39.966053 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6f964-a8ff-4809-a03f-f3fc1d13c0bc None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:40.025412 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef88770-6674-4bf8-b87d-690dea537e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:40.026070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef88770-6674-4bf8-b87d-690dea537e3f None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:40.026418 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef88770-6674-4bf8-b87d-690dea537e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:40.026780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef88770-6674-4bf8-b87d-690dea537e3f None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:40.027515 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef88770-6674-4bf8-b87d-690dea537e3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:40.028494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef88770-6674-4bf8-b87d-690dea537e3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:40.031639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef88770-6674-4bf8-b87d-690dea537e3f None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:40.044578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 210/411] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:40.046055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30206967-af82-4f07-9f5e-048f45a1e3c3 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:40.046426 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30206967-af82-4f07-9f5e-048f45a1e3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:40.046939 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30206967-af82-4f07-9f5e-048f45a1e3c3 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:40.048336 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 211/412] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:40.054144 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30206967-af82-4f07-9f5e-048f45a1e3c3 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:40.056150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:40.092859 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc2d29-fb27-42df-9f47-b7dec1d75375 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:40.134648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74fc3e-b213-4bac-b9ee-a507ae9eb2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:40.135318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74fc3e-b213-4bac-b9ee-a507ae9eb2a0 None admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:40.135598 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74fc3e-b213-4bac-b9ee-a507ae9eb2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:40.135877 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74fc3e-b213-4bac-b9ee-a507ae9eb2a0 None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222/groups/87f7784d531244e9963fa10fbbf8ed8f/roles/b9184599d95b4950beca34304cdd2eb8` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:40.139551 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f74fc3e-b213-4bac-b9ee-a507ae9eb2a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e1ab10529d34aaabbf5bdd2cff84222, group_id=87f7784d531244e9963fa10fbbf8ed8f, role_id=b9184599d95b4950beca34304cdd2eb8)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:40.140991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f74fc3e-b213-4bac-b9ee-a507ae9eb2a0 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:40.158125 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 202/413] 10.222.0.73 () {58 vars in 1442 bytes} [Mon Aug 29 19:59:39 2022] PUT /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222/groups/87f7784d531244e9963fa10fbbf8ed8f/roles/b9184599d95b4950beca34304cdd2eb8 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:59:40.163088 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb96201-accf-49b1-bf4f-7651262bc8b9 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:40.163088 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb96201-accf-49b1-bf4f-7651262bc8b9 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:40.163088 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb96201-accf-49b1-bf4f-7651262bc8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:40.182869 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-9eb96201-accf-49b1-bf4f-7651262bc8b9 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:40.184732 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9eb96201-accf-49b1-bf4f-7651262bc8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:40.215413 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30206967-af82-4f07-9f5e-048f45a1e3c3 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:40.263512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:40.271805 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 203/414] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:40.295772 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6438a-6c2b-4bff-96da-f0c0c661f722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:40.296781 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6438a-6c2b-4bff-96da-f0c0c661f722 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:40.297260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6438a-6c2b-4bff-96da-f0c0c661f722 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:40.297750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6438a-6c2b-4bff-96da-f0c0c661f722 None admin] PATH_INFO: `/v3/groups/dff53ed4f9a94eee82f8d8ef3b0c098d` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:40.299083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6438a-6c2b-4bff-96da-f0c0c661f722 None admin] RBAC: Authorizing `identity:get_group(group_id=dff53ed4f9a94eee82f8d8ef3b0c098d)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:40.300714 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6438a-6c2b-4bff-96da-f0c0c661f722 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:40.302620 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 212/415] 10.222.0.73 () {56 vars in 1180 bytes} [Mon Aug 29 19:59:40 2022] GET /identity/v3/groups/dff53ed4f9a94eee82f8d8ef3b0c098d => generated 229 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:40.308544 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb96201-accf-49b1-bf4f-7651262bc8b9 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:40.314132 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f3aa4-107a-4383-af39-6093174b6713 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:40.314798 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f3aa4-107a-4383-af39-6093174b6713 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:40.315269 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f3aa4-107a-4383-af39-6093174b6713 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:40.316793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 213/416] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:59:40.322259 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c155bd5-c26a-4699-9af1-7881841d576d None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:40.322839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c155bd5-c26a-4699-9af1-7881841d576d None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:40.323308 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c155bd5-c26a-4699-9af1-7881841d576d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:40.354491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-3c155bd5-c26a-4699-9af1-7881841d576d None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:40.356813 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-3c155bd5-c26a-4699-9af1-7881841d576d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:40.381087 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:40.441969 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb96201-accf-49b1-bf4f-7651262bc8b9 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:40.495857 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede392e-0b04-4322-8f6c-d0de5360a3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:40.498141 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede392e-0b04-4322-8f6c-d0de5360a3a8 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:40.498625 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede392e-0b04-4322-8f6c-d0de5360a3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:40.499109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede392e-0b04-4322-8f6c-d0de5360a3a8 None admin] PATH_INFO: `/v3/projects/ee34b4bd3d8043258a1a5d3588fa8f9c` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:40.500497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede392e-0b04-4322-8f6c-d0de5360a3a8 None admin] RBAC: Authorizing `identity:get_project(project_id=ee34b4bd3d8043258a1a5d3588fa8f9c)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:40.502151 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede392e-0b04-4322-8f6c-d0de5360a3a8 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:40.504298 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 204/417] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:40 2022] GET /identity/v3/projects/ee34b4bd3d8043258a1a5d3588fa8f9c => generated 344 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:40.509941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede392e-0b04-4322-8f6c-d0de5360a3a8 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:40.516617 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 214/418] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:59:40.579423 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c155bd5-c26a-4699-9af1-7881841d576d None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:40.654775 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede392e-0b04-4322-8f6c-d0de5360a3a8 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:40.699690 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:40.707206 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c155bd5-c26a-4699-9af1-7881841d576d None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:40.723682 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ee16f-c7f7-43d3-bf78-9c8369c32332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:40.724763 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ee16f-c7f7-43d3-bf78-9c8369c32332 None admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:40.724763 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ee16f-c7f7-43d3-bf78-9c8369c32332 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:40.725067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ee16f-c7f7-43d3-bf78-9c8369c32332 None admin] PATH_INFO: `/v3/projects/ee34b4bd3d8043258a1a5d3588fa8f9c/groups/dff53ed4f9a94eee82f8d8ef3b0c098d/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:40.731604 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0ee16f-c7f7-43d3-bf78-9c8369c32332 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee34b4bd3d8043258a1a5d3588fa8f9c, group_id=dff53ed4f9a94eee82f8d8ef3b0c098d, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:40.733066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0ee16f-c7f7-43d3-bf78-9c8369c32332 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:40.759765 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20cd8c8-24cd-4a65-a841-9499ccd183e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:40.760254 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 205/419] 10.222.0.73 () {58 vars in 1442 bytes} [Mon Aug 29 19:59:40 2022] PUT /identity/v3/projects/ee34b4bd3d8043258a1a5d3588fa8f9c/groups/dff53ed4f9a94eee82f8d8ef3b0c098d/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:59:40.761047 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cd8c8-24cd-4a65-a841-9499ccd183e9 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:40.761486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cd8c8-24cd-4a65-a841-9499ccd183e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:40.762252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cd8c8-24cd-4a65-a841-9499ccd183e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:40.782046 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:40.782982 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20cd8c8-24cd-4a65-a841-9499ccd183e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:40.788452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-e20cd8c8-24cd-4a65-a841-9499ccd183e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:40.788452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:40.788452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:40.788452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:40.788452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:40.788452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:40.788452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:40.788452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:40.788452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:40.788452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:40.788452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:40.788452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:40.788452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:40.788452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:40.788452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:40.788452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:40.789378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:40.789378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:40.789378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:40.789378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:40.789378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:40.789378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:40.789378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:40.789378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:40.789378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:40.789378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:40.789378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:40.789378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:40.789378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:40.789378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:40.789378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:40.789378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:40.790237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:40.790237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:40.790237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:40.790237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:40.790237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:40.790237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:40.790237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:40.790237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:40.790237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:40.790237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:40.790237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:40.790237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:40.792264 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 215/420] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:40.799074 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ee16f-c7f7-43d3-bf78-9c8369c32332 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:40.931580 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20cd8c8-24cd-4a65-a841-9499ccd183e9 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:40.951399 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ee16f-c7f7-43d3-bf78-9c8369c32332 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:40.994578 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd9b0b-0ede-44a6-9fc7-62ecf65ee154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:40.995593 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd9b0b-0ede-44a6-9fc7-62ecf65ee154 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:40.996078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd9b0b-0ede-44a6-9fc7-62ecf65ee154 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:40.997029 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd9b0b-0ede-44a6-9fc7-62ecf65ee154 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:40.998576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd9b0b-0ede-44a6-9fc7-62ecf65ee154 None admin] RBAC: Adding query filter params (name=member) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:40.999236 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd9b0b-0ede-44a6-9fc7-62ecf65ee154 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:41.001540 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd9b0b-0ede-44a6-9fc7-62ecf65ee154 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:41.011818 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 206/421] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:41.018698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd9b0b-0ede-44a6-9fc7-62ecf65ee154 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:41.096875 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:41.139720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20cd8c8-24cd-4a65-a841-9499ccd183e9 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:41.160971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd9b0b-0ede-44a6-9fc7-62ecf65ee154 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:41.224010 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9664f0-5a06-4afd-be67-ad765dd99e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:41.224549 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9664f0-5a06-4afd-be67-ad765dd99e09 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:41.224830 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9664f0-5a06-4afd-be67-ad765dd99e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:41.225102 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9664f0-5a06-4afd-be67-ad765dd99e09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:41.241700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107ddb1-394f-45bb-9693-f7991da001d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:41.247267 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107ddb1-394f-45bb-9693-f7991da001d6 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:41.247616 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107ddb1-394f-45bb-9693-f7991da001d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:41.247937 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107ddb1-394f-45bb-9693-f7991da001d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:41.271937 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107ddb1-394f-45bb-9693-f7991da001d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:41.277175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-9107ddb1-394f-45bb-9693-f7991da001d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:41.277175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:41.277175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:41.277175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:41.277175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:41.277175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:41.277175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:41.277175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:41.277175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:41.277175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:41.277175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:41.277175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:41.277175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:41.277175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:41.277175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:41.277175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:41.278268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:41.278268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:41.278268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:41.278268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:41.278268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:41.278268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:41.278268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:41.278268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:41.278268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:41.278268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:41.278268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:41.278268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:41.278268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:41.278268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:41.278268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:41.278268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:41.279478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:41.279478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:41.279478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:41.279478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:41.279478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:41.279478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:41.279478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:41.279478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:41.279478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:41.279478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:41.279478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:41.279478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:41.279478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 216/422] 10.222.0.73 () {56 vars in 1114 bytes} [Mon Aug 29 19:59:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 19:59:41.286039 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107ddb1-394f-45bb-9693-f7991da001d6 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:41.341634 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:41.409835 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107ddb1-394f-45bb-9693-f7991da001d6 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:41.447752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4498727-aa4d-417a-82d2-8a0136d4aa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:41.448413 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4498727-aa4d-417a-82d2-8a0136d4aa29 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:41.448690 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4498727-aa4d-417a-82d2-8a0136d4aa29 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:41.448965 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4498727-aa4d-417a-82d2-8a0136d4aa29 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:41.449850 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4498727-aa4d-417a-82d2-8a0136d4aa29 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:41.450283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4498727-aa4d-417a-82d2-8a0136d4aa29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:41.452224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4498727-aa4d-417a-82d2-8a0136d4aa29 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:41.461805 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 217/423] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 19:59:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:41.467478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4498727-aa4d-417a-82d2-8a0136d4aa29 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:41.475911 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9664f0-5a06-4afd-be67-ad765dd99e09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:41.477311 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9664f0-5a06-4afd-be67-ad765dd99e09 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:41.576191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4498727-aa4d-417a-82d2-8a0136d4aa29 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:41.577790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:41.585006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:41.645431 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540919b5-f539-4361-b130-97b0984a9755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:41.646234 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540919b5-f539-4361-b130-97b0984a9755 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:41.646516 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540919b5-f539-4361-b130-97b0984a9755 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:41.647049 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540919b5-f539-4361-b130-97b0984a9755 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:41.670023 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 207/424] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 652 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:41.671420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2c521-b068-4955-b15a-ee83784fdc92 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:41.671706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2c521-b068-4955-b15a-ee83784fdc92 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:41.671973 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2c521-b068-4955-b15a-ee83784fdc92 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:41.673394 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 208/425] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:41.676219 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2c521-b068-4955-b15a-ee83784fdc92 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:41.733444 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540919b5-f539-4361-b130-97b0984a9755 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:41.734921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540919b5-f539-4361-b130-97b0984a9755 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:41.788806 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2c521-b068-4955-b15a-ee83784fdc92 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:41.827776 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75366721-ab5a-4ac3-976e-a775c4869cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:41.828416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75366721-ab5a-4ac3-976e-a775c4869cdd None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:41.828694 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75366721-ab5a-4ac3-976e-a775c4869cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:41.828968 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75366721-ab5a-4ac3-976e-a775c4869cdd None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:41.830264 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75366721-ab5a-4ac3-976e-a775c4869cdd None admin] RBAC: Authorizing `identity:get_project(project_id=4367ec2ce611409a85975a2a505f1ff8)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:41.832550 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75366721-ab5a-4ac3-976e-a775c4869cdd None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:41.837659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 218/427] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:41.839035 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e558d-4335-4e5e-9dcc-5a77e97ee90b None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:41.839336 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e558d-4335-4e5e-9dcc-5a77e97ee90b None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:41.839674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 209/427] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:41 2022] GET /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:41.839863 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e558d-4335-4e5e-9dcc-5a77e97ee90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:41.840673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75366721-ab5a-4ac3-976e-a775c4869cdd None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:41.846970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:41.861358 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-2c9e558d-4335-4e5e-9dcc-5a77e97ee90b None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:41.862733 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-2c9e558d-4335-4e5e-9dcc-5a77e97ee90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:41.867984 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:41.937986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 219/428] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:41.946692 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e558d-4335-4e5e-9dcc-5a77e97ee90b None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:41.956942 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75366721-ab5a-4ac3-976e-a775c4869cdd None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:42.011590 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413b411-903e-4fdd-bd4b-0b0443d79560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:42.012631 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413b411-903e-4fdd-bd4b-0b0443d79560 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:42.013057 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413b411-903e-4fdd-bd4b-0b0443d79560 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:42.013473 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413b411-903e-4fdd-bd4b-0b0443d79560 None admin] PATH_INFO: `/v3/groups/87f7784d531244e9963fa10fbbf8ed8f` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:42.014844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413b411-903e-4fdd-bd4b-0b0443d79560 None admin] RBAC: Authorizing `identity:get_group(group_id=87f7784d531244e9963fa10fbbf8ed8f)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:42.016442 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413b411-903e-4fdd-bd4b-0b0443d79560 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:42.018491 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 210/429] 10.222.0.73 () {56 vars in 1180 bytes} [Mon Aug 29 19:59:41 2022] GET /identity/v3/groups/87f7784d531244e9963fa10fbbf8ed8f => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:42.031669 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413b411-903e-4fdd-bd4b-0b0443d79560 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:42.111966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e558d-4335-4e5e-9dcc-5a77e97ee90b None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:42.146817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:42.154866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42abc44b-1175-4473-bc73-a6a76854a5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:42.155506 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abc44b-1175-4473-bc73-a6a76854a5c9 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:42.155788 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abc44b-1175-4473-bc73-a6a76854a5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:42.156063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abc44b-1175-4473-bc73-a6a76854a5c9 None admin] PATH_INFO: `/v3/projects/4367ec2ce611409a85975a2a505f1ff8` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:42.157301 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42abc44b-1175-4473-bc73-a6a76854a5c9 None admin] RBAC: Authorizing `identity:get_project(project_id=4367ec2ce611409a85975a2a505f1ff8)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:42.159187 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42abc44b-1175-4473-bc73-a6a76854a5c9 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:42.160830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 220/430] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:41 2022] GET /identity/v3/projects/4367ec2ce611409a85975a2a505f1ff8 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:42.166644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42abc44b-1175-4473-bc73-a6a76854a5c9 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:42.234192 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:42.269750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413b411-903e-4fdd-bd4b-0b0443d79560 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:42.302720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42abc44b-1175-4473-bc73-a6a76854a5c9 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:42.342450 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd3a95-b9d1-4556-a352-e608f07decd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:42.343392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd3a95-b9d1-4556-a352-e608f07decd5 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:42.343669 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd3a95-b9d1-4556-a352-e608f07decd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:42.343939 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd3a95-b9d1-4556-a352-e608f07decd5 None admin] PATH_INFO: `/v3/groups/87f7784d531244e9963fa10fbbf8ed8f` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:42.345819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd3a95-b9d1-4556-a352-e608f07decd5 None admin] RBAC: Authorizing `identity:get_group(group_id=87f7784d531244e9963fa10fbbf8ed8f)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:42.348121 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd3a95-b9d1-4556-a352-e608f07decd5 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:42.349887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 221/431] 10.222.0.73 () {56 vars in 1180 bytes} [Mon Aug 29 19:59:42 2022] GET /identity/v3/groups/87f7784d531244e9963fa10fbbf8ed8f => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:42.350501 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd3a95-b9d1-4556-a352-e608f07decd5 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:42.355011 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f63d4-9c41-4144-9515-f7320ce1a4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:42.356825 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f63d4-9c41-4144-9515-f7320ce1a4de None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:42.357110 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f63d4-9c41-4144-9515-f7320ce1a4de None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:42.357380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f63d4-9c41-4144-9515-f7320ce1a4de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:42.358145 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f63d4-9c41-4144-9515-f7320ce1a4de None admin] RBAC: Adding query filter params (group.id=87f7784d531244e9963fa10fbbf8ed8f, role.id=9265e1fd617b4c0b8fbf267885394060, scope.project.id=4367ec2ce611409a85975a2a505f1ff8) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:42.358450 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f63d4-9c41-4144-9515-f7320ce1a4de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:42.359873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f63d4-9c41-4144-9515-f7320ce1a4de None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:42.371068 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 211/432] 10.222.0.73 () {56 vars in 1376 bytes} [Mon Aug 29 19:59:42 2022] GET /identity/v3/role_assignments?group.id=87f7784d531244e9963fa10fbbf8ed8f&scope.project.id=4367ec2ce611409a85975a2a505f1ff8&role.id=9265e1fd617b4c0b8fbf267885394060 => generated 611 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:42.372992 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f63d4-9c41-4144-9515-f7320ce1a4de None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:42.511383 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:42.551473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd3a95-b9d1-4556-a352-e608f07decd5 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:42.567766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f63d4-9c41-4144-9515-f7320ce1a4de None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:42.593653 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9311737-6c8b-47eb-a995-19eb33e30758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:42.594492 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9311737-6c8b-47eb-a995-19eb33e30758 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:42.594603 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9311737-6c8b-47eb-a995-19eb33e30758 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:42.594880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9311737-6c8b-47eb-a995-19eb33e30758 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:42.606605 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9311737-6c8b-47eb-a995-19eb33e30758 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:42.612524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-b9311737-6c8b-47eb-a995-19eb33e30758 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:42.612524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:42.612524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:42.612524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:42.612524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:42.612524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:42.612524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:42.612524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:42.612524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:42.612524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:42.612524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:42.612524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:42.612524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:42.612524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:42.612524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:42.612524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:42.613453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:42.613453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:42.613453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:42.613453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:42.613453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:42.613453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:42.613453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:42.613453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:42.613453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:42.613453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:42.613453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:42.613453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:42.613453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:42.613453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:42.613453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:42.613453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:42.614342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:42.614342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:42.614342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:42.614342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:42.614342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:42.614342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:42.614342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:42.614342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:42.614342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:42.614342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:42.614342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 19:59:42.614342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:42.614342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 222/433] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:42.615000 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8e066-e7a9-42e0-8aa6-f937302c74ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:42.615094 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de27532-4257-4561-a7d2-638ebd9696e6 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:42.615094 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de27532-4257-4561-a7d2-638ebd9696e6 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:42.615214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8e066-e7a9-42e0-8aa6-f937302c74ee None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:42.615277 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de27532-4257-4561-a7d2-638ebd9696e6 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:42.615341 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8e066-e7a9-42e0-8aa6-f937302c74ee None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:42.615341 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8e066-e7a9-42e0-8aa6-f937302c74ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:42.617739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8e066-e7a9-42e0-8aa6-f937302c74ee None admin] RBAC: Adding query filter params (group.id=87f7784d531244e9963fa10fbbf8ed8f, role.id=b9184599d95b4950beca34304cdd2eb8, scope.project.id=4367ec2ce611409a85975a2a505f1ff8) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:42.617860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 223/434] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:42.617948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8e066-e7a9-42e0-8aa6-f937302c74ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:42.617948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8e066-e7a9-42e0-8aa6-f937302c74ee None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:42.622276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad8250-4491-424a-9eaf-3cc2482ed49d None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:42.622559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad8250-4491-424a-9eaf-3cc2482ed49d None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:42.622924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad8250-4491-424a-9eaf-3cc2482ed49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:42.627667 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 212/435] 10.222.0.73 () {56 vars in 1376 bytes} [Mon Aug 29 19:59:42 2022] GET /identity/v3/role_assignments?group.id=87f7784d531244e9963fa10fbbf8ed8f&scope.project.id=4367ec2ce611409a85975a2a505f1ff8&role.id=b9184599d95b4950beca34304cdd2eb8 => generated 611 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:42.628793 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8e066-e7a9-42e0-8aa6-f937302c74ee None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:42.643816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-60ad8250-4491-424a-9eaf-3cc2482ed49d None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:42.645217 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-60ad8250-4491-424a-9eaf-3cc2482ed49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:42.662138 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:42.748282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 224/436] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:59:42.793158 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8e066-e7a9-42e0-8aa6-f937302c74ee None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:42.838909 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e10001-a8d0-4cf2-b3ad-ed439336d585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:42.839529 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e10001-a8d0-4cf2-b3ad-ed439336d585 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:42.839810 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e10001-a8d0-4cf2-b3ad-ed439336d585 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:42.840082 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e10001-a8d0-4cf2-b3ad-ed439336d585 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:42.840798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e10001-a8d0-4cf2-b3ad-ed439336d585 None admin] RBAC: Adding query filter params (name=member) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:42.841102 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e10001-a8d0-4cf2-b3ad-ed439336d585 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:42.842535 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e10001-a8d0-4cf2-b3ad-ed439336d585 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:42.851933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 213/437] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 19:59:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:42.858063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad8250-4491-424a-9eaf-3cc2482ed49d None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:42.924946 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:42.958862 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd434c-df94-47ce-84f0-f719820f15a8 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:42.959231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd434c-df94-47ce-84f0-f719820f15a8 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:42.959420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd434c-df94-47ce-84f0-f719820f15a8 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:42.961358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 214/438] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:59:42.968199 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57830e65-7c3c-4173-bab9-95e391c59cd9 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:42.968758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57830e65-7c3c-4173-bab9-95e391c59cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:42.968972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57830e65-7c3c-4173-bab9-95e391c59cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:42.979231 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad8250-4491-424a-9eaf-3cc2482ed49d None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:42.993529 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-57830e65-7c3c-4173-bab9-95e391c59cd9 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:42.994938 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-57830e65-7c3c-4173-bab9-95e391c59cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:43.016936 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035c36c-17f4-4e1c-a4af-fce12d9f3033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:43.017572 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035c36c-17f4-4e1c-a4af-fce12d9f3033 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:43.017886 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035c36c-17f4-4e1c-a4af-fce12d9f3033 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:43.018160 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035c36c-17f4-4e1c-a4af-fce12d9f3033 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:43.088756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 215/439] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:59:43.089006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57830e65-7c3c-4173-bab9-95e391c59cd9 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:43.102103 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:43.113301 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035c36c-17f4-4e1c-a4af-fce12d9f3033 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:43.114864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035c36c-17f4-4e1c-a4af-fce12d9f3033 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:43.195163 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:43.226785 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57830e65-7c3c-4173-bab9-95e391c59cd9 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:43.255954 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 225/440] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:43.261610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035c36c-17f4-4e1c-a4af-fce12d9f3033 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:43.276523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8879aa-89a7-4a50-979c-0a0cf0a8e143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:43.277171 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8879aa-89a7-4a50-979c-0a0cf0a8e143 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:43.277437 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8879aa-89a7-4a50-979c-0a0cf0a8e143 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:43.277837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8879aa-89a7-4a50-979c-0a0cf0a8e143 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:43.291565 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8879aa-89a7-4a50-979c-0a0cf0a8e143 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:43.298647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-9f8879aa-89a7-4a50-979c-0a0cf0a8e143 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:43.298647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:43.298647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:43.298647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:43.298647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:43.298647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:43.298647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:43.298647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:43.298647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:43.298647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:43.298647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:43.298647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:43.298647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:43.298647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:43.298647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:43.298647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:43.299576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:43.299576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:43.299576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:43.299576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:43.299576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:43.299576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:43.299576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:43.299576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:43.299576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:43.299576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:43.299576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:43.299576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:43.299576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:43.299576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:43.299576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:43.299576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:43.300364 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:43.300364 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:43.300364 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:43.300364 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:43.300364 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:43.300364 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:43.300364 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:43.300364 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:43.300364 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:43.300364 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:43.300364 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:43.300364 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:43.300951 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 216/441] 10.222.0.73 () {56 vars in 1114 bytes} [Mon Aug 29 19:59:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:43.306340 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8879aa-89a7-4a50-979c-0a0cf0a8e143 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:43.382162 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035c36c-17f4-4e1c-a4af-fce12d9f3033 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:43.383768 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:43.415813 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c0c2d-b068-42be-b079-60c05e9522cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:43.416439 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c0c2d-b068-42be-b079-60c05e9522cc None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:43.416715 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c0c2d-b068-42be-b079-60c05e9522cc None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:43.416985 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c0c2d-b068-42be-b079-60c05e9522cc None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:43.417613 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8879aa-89a7-4a50-979c-0a0cf0a8e143 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:43.418138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c0c2d-b068-42be-b079-60c05e9522cc None admin] RBAC: Authorizing `identity:get_project(project_id=2e1ab10529d34aaabbf5bdd2cff84222)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:43.420319 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c0c2d-b068-42be-b079-60c05e9522cc None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:43.421977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 226/442] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:43 2022] GET /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:43.423820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c0c2d-b068-42be-b079-60c05e9522cc None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:43.453005 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:43.454905 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3affbb96-1205-4da8-b767-e52c22608230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:43.455517 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affbb96-1205-4da8-b767-e52c22608230 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:43.455856 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affbb96-1205-4da8-b767-e52c22608230 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:43.456192 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affbb96-1205-4da8-b767-e52c22608230 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:43.456917 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3affbb96-1205-4da8-b767-e52c22608230 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:43.457221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3affbb96-1205-4da8-b767-e52c22608230 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:43.458690 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3affbb96-1205-4da8-b767-e52c22608230 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:43.467548 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 217/443] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 19:59:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:43.468354 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3affbb96-1205-4da8-b767-e52c22608230 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:43.534888 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c0c2d-b068-42be-b079-60c05e9522cc None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:43.569365 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368dec3-894d-424d-bfad-8c918943eeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:43.569564 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3affbb96-1205-4da8-b767-e52c22608230 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:43.570027 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368dec3-894d-424d-bfad-8c918943eeea None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:43.570306 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368dec3-894d-424d-bfad-8c918943eeea None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:43.570579 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368dec3-894d-424d-bfad-8c918943eeea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:43.581810 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4368dec3-894d-424d-bfad-8c918943eeea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:43.587101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-4368dec3-894d-424d-bfad-8c918943eeea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:59:43.587101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:43.587101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:43.587101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:43.587101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:43.587101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:43.587101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:43.587101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:43.587101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:43.587101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:43.587101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:43.587101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:43.587101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:43.587101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:43.587101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:43.587101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:43.588010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:43.588010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:43.588010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:43.588010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:43.588010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:43.588010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:43.588010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:43.588010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:43.588010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:43.588010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:43.588010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:43.588010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:43.588010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:43.588010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:43.588010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:43.588010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:43.588787 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:43.588787 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:43.588787 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:43.588787 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:43.588787 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:43.588787 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:43.588787 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:43.588787 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:43.588787 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:43.588787 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:43.588787 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 19:59:43.588787 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:43.588787 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 227/444] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 19:59:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:43.589435 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368dec3-894d-424d-bfad-8c918943eeea None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:43.601766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:43.608815 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a5c58-28d1-41be-a078-8339098c2550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:43.609504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a5c58-28d1-41be-a078-8339098c2550 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:43.609821 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a5c58-28d1-41be-a078-8339098c2550 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:43.610124 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a5c58-28d1-41be-a078-8339098c2550 None admin] PATH_INFO: `/v3/groups/87f7784d531244e9963fa10fbbf8ed8f` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:43.611266 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a5c58-28d1-41be-a078-8339098c2550 None admin] RBAC: Authorizing `identity:get_group(group_id=87f7784d531244e9963fa10fbbf8ed8f)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:43.612675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a5c58-28d1-41be-a078-8339098c2550 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:43.614272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 218/445] 10.222.0.73 () {56 vars in 1180 bytes} [Mon Aug 29 19:59:43 2022] GET /identity/v3/groups/87f7784d531244e9963fa10fbbf8ed8f => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:43.615024 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a5c58-28d1-41be-a078-8339098c2550 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:43.680149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:43.695566 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368dec3-894d-424d-bfad-8c918943eeea None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:43.717072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a5c58-28d1-41be-a078-8339098c2550 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:43.729891 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d47c9-31c8-469a-9c23-61b12e62db94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:43.730505 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d47c9-31c8-469a-9c23-61b12e62db94 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:43.730829 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d47c9-31c8-469a-9c23-61b12e62db94 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:43.731085 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d47c9-31c8-469a-9c23-61b12e62db94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:43.751631 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312140c-992e-4e75-9082-c85e2fbb718f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:43.752268 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312140c-992e-4e75-9082-c85e2fbb718f None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:43.752552 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312140c-992e-4e75-9082-c85e2fbb718f None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:43.752827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312140c-992e-4e75-9082-c85e2fbb718f None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:43.753489 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312140c-992e-4e75-9082-c85e2fbb718f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:43.753834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312140c-992e-4e75-9082-c85e2fbb718f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:43.755217 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312140c-992e-4e75-9082-c85e2fbb718f None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:43.764182 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 219/446] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:43.764995 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312140c-992e-4e75-9082-c85e2fbb718f None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:43.802502 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d47c9-31c8-469a-9c23-61b12e62db94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:43.803979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d47c9-31c8-469a-9c23-61b12e62db94 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:43.823893 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:43.869227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312140c-992e-4e75-9082-c85e2fbb718f None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:43.892900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:43.904718 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a903b-8de3-4839-b30b-cd8fb7665254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:43.905327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a903b-8de3-4839-b30b-cd8fb7665254 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:43.905610 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a903b-8de3-4839-b30b-cd8fb7665254 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:43.906032 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a903b-8de3-4839-b30b-cd8fb7665254 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:43.906758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a903b-8de3-4839-b30b-cd8fb7665254 None admin] RBAC: Adding query filter params (group.id=87f7784d531244e9963fa10fbbf8ed8f, role.id=9265e1fd617b4c0b8fbf267885394060, scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:43.907069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a903b-8de3-4839-b30b-cd8fb7665254 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:43.908548 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a903b-8de3-4839-b30b-cd8fb7665254 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:43.909360 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 228/447] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:43.910225 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d47c9-31c8-469a-9c23-61b12e62db94 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:43.917945 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 220/448] 10.222.0.73 () {56 vars in 1376 bytes} [Mon Aug 29 19:59:43 2022] GET /identity/v3/role_assignments?group.id=87f7784d531244e9963fa10fbbf8ed8f&scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222&role.id=9265e1fd617b4c0b8fbf267885394060 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:43.917945 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a903b-8de3-4839-b30b-cd8fb7665254 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:44.023027 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d47c9-31c8-469a-9c23-61b12e62db94 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:44.038283 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a903b-8de3-4839-b30b-cd8fb7665254 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:44.063477 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec80e5-2285-4479-a0d8-6f13fb9c0322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:44.064434 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec80e5-2285-4479-a0d8-6f13fb9c0322 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:44.064716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec80e5-2285-4479-a0d8-6f13fb9c0322 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:44.064990 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec80e5-2285-4479-a0d8-6f13fb9c0322 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:44.066012 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:44.079866 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcb429-ab11-4531-9573-d05f0d63d891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:44.080823 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcb429-ab11-4531-9573-d05f0d63d891 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:44.081357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcb429-ab11-4531-9573-d05f0d63d891 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:44.081541 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcb429-ab11-4531-9573-d05f0d63d891 None admin] PATH_INFO: `/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:44.083527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdcb429-ab11-4531-9573-d05f0d63d891 None admin] RBAC: Authorizing `identity:get_project(project_id=2e1ab10529d34aaabbf5bdd2cff84222)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:44.085027 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdcb429-ab11-4531-9573-d05f0d63d891 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:44.087089 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 221/449] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:43 2022] GET /identity/v3/projects/2e1ab10529d34aaabbf5bdd2cff84222 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:44.093319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcb429-ab11-4531-9573-d05f0d63d891 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:44.144475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:44.160594 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec80e5-2285-4479-a0d8-6f13fb9c0322 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:44.162203 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec80e5-2285-4479-a0d8-6f13fb9c0322 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:44.218504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcb429-ab11-4531-9573-d05f0d63d891 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:44.260055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f521ec-a2d0-4f64-8b72-33af1c5168c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:44.260055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f521ec-a2d0-4f64-8b72-33af1c5168c7 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:44.260055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f521ec-a2d0-4f64-8b72-33af1c5168c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:44.260055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f521ec-a2d0-4f64-8b72-33af1c5168c7 None admin] PATH_INFO: `/v3/groups/87f7784d531244e9963fa10fbbf8ed8f` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:44.260808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f521ec-a2d0-4f64-8b72-33af1c5168c7 None admin] RBAC: Authorizing `identity:get_group(group_id=87f7784d531244e9963fa10fbbf8ed8f)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:44.262893 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f521ec-a2d0-4f64-8b72-33af1c5168c7 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:44.264536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 222/450] 10.222.0.73 () {56 vars in 1180 bytes} [Mon Aug 29 19:59:44 2022] GET /identity/v3/groups/87f7784d531244e9963fa10fbbf8ed8f => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:44.271063 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f521ec-a2d0-4f64-8b72-33af1c5168c7 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:44.290657 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 229/451] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:44.296404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec80e5-2285-4479-a0d8-6f13fb9c0322 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:44.330970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:44.372721 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f521ec-a2d0-4f64-8b72-33af1c5168c7 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:44.381845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:44.398482 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec80e5-2285-4479-a0d8-6f13fb9c0322 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:44.408251 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0eb15-18c9-4817-a0cc-8e4324e911cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:44.408855 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0eb15-18c9-4817-a0cc-8e4324e911cd None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:44.409138 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0eb15-18c9-4817-a0cc-8e4324e911cd None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:44.409414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0eb15-18c9-4817-a0cc-8e4324e911cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:44.410165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e0eb15-18c9-4817-a0cc-8e4324e911cd None admin] RBAC: Adding query filter params (group.id=87f7784d531244e9963fa10fbbf8ed8f, role.id=b9184599d95b4950beca34304cdd2eb8, scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:44.410478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e0eb15-18c9-4817-a0cc-8e4324e911cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:44.411900 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e0eb15-18c9-4817-a0cc-8e4324e911cd None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:44.417729 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 223/452] 10.222.0.73 () {56 vars in 1376 bytes} [Mon Aug 29 19:59:44 2022] GET /identity/v3/role_assignments?group.id=87f7784d531244e9963fa10fbbf8ed8f&scope.project.id=2e1ab10529d34aaabbf5bdd2cff84222&role.id=b9184599d95b4950beca34304cdd2eb8 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:44.432775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76fda6-483e-479f-a247-17aa3053991f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:44.433421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76fda6-483e-479f-a247-17aa3053991f None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:44.433733 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76fda6-483e-479f-a247-17aa3053991f None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:44.434504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76fda6-483e-479f-a247-17aa3053991f None admin] PATH_INFO: `/v3/projects/ee34b4bd3d8043258a1a5d3588fa8f9c` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:44.436102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76fda6-483e-479f-a247-17aa3053991f None admin] RBAC: Authorizing `identity:get_project(project_id=ee34b4bd3d8043258a1a5d3588fa8f9c)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:44.437602 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76fda6-483e-479f-a247-17aa3053991f None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:44.439372 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 230/453] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 19:59:44 2022] GET /identity/v3/projects/ee34b4bd3d8043258a1a5d3588fa8f9c => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:44.445426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0eb15-18c9-4817-a0cc-8e4324e911cd None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:44.563050 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0eb15-18c9-4817-a0cc-8e4324e911cd None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:44.587129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:44.598355 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa22bf-2c2d-44e8-b99e-b856371bea07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:44.598994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa22bf-2c2d-44e8-b99e-b856371bea07 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:44.599280 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa22bf-2c2d-44e8-b99e-b856371bea07 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:44.599558 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa22bf-2c2d-44e8-b99e-b856371bea07 None admin] PATH_INFO: `/v3/groups/dff53ed4f9a94eee82f8d8ef3b0c098d` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:44.600697 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa22bf-2c2d-44e8-b99e-b856371bea07 None admin] RBAC: Authorizing `identity:get_group(group_id=dff53ed4f9a94eee82f8d8ef3b0c098d)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:44.602192 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa22bf-2c2d-44e8-b99e-b856371bea07 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:44.603788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 224/454] 10.222.0.73 () {56 vars in 1180 bytes} [Mon Aug 29 19:59:44 2022] GET /identity/v3/groups/dff53ed4f9a94eee82f8d8ef3b0c098d => generated 229 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:44.609931 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76fda6-483e-479f-a247-17aa3053991f None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:44.714642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76fda6-483e-479f-a247-17aa3053991f None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:44.750286 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e97cd5-cacd-4729-83b0-991cc83d8211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:44.750887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e97cd5-cacd-4729-83b0-991cc83d8211 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:44.751166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e97cd5-cacd-4729-83b0-991cc83d8211 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:44.751437 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e97cd5-cacd-4729-83b0-991cc83d8211 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:44.752158 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e97cd5-cacd-4729-83b0-991cc83d8211 None admin] RBAC: Adding query filter params (group.id=dff53ed4f9a94eee82f8d8ef3b0c098d, role.id=296ee2c194824485af71618619f82e09, scope.project.id=ee34b4bd3d8043258a1a5d3588fa8f9c) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:44.752465 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e97cd5-cacd-4729-83b0-991cc83d8211 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:44.753917 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e97cd5-cacd-4729-83b0-991cc83d8211 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:44.759906 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 231/455] 10.222.0.73 () {56 vars in 1376 bytes} [Mon Aug 29 19:59:44 2022] GET /identity/v3/role_assignments?group.id=dff53ed4f9a94eee82f8d8ef3b0c098d&scope.project.id=ee34b4bd3d8043258a1a5d3588fa8f9c&role.id=296ee2c194824485af71618619f82e09 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:47.096724 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22b06a-f302-423c-aab8-e04e70d869e8 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:47.096724 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22b06a-f302-423c-aab8-e04e70d869e8 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:47.096724 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22b06a-f302-423c-aab8-e04e70d869e8 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:47.098357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 225/456] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:47.104359 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b8799-fd69-4450-99c4-9329d0606c01 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:47.105246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b8799-fd69-4450-99c4-9329d0606c01 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:47.105752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b8799-fd69-4450-99c4-9329d0606c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:47.134845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-c42b8799-fd69-4450-99c4-9329d0606c01 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:47.136027 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c42b8799-fd69-4450-99c4-9329d0606c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:47.158089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:47.235999 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 232/457] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:59:47.290618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24ed3d-37ab-4277-bc42-8e8afce904cd None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:47.290963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24ed3d-37ab-4277-bc42-8e8afce904cd None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:47.291220 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24ed3d-37ab-4277-bc42-8e8afce904cd None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:47.292809 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 226/458] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:47.298046 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7986c-0ee5-4b37-8b46-32f87daf18ee None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:47.298369 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7986c-0ee5-4b37-8b46-32f87daf18ee None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:47.298644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7986c-0ee5-4b37-8b46-32f87daf18ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:47.327054 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-4af7986c-0ee5-4b37-8b46-32f87daf18ee None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:47.328251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-4af7986c-0ee5-4b37-8b46-32f87daf18ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:47.420326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 233/459] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:47.545115 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24ed3d-37ab-4277-bc42-8e8afce904cd None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:47.664242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24ed3d-37ab-4277-bc42-8e8afce904cd None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:47.698388 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e000837-8bdc-4b48-9d5a-1d0ee441b783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:47.698944 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e000837-8bdc-4b48-9d5a-1d0ee441b783 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:47.699225 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e000837-8bdc-4b48-9d5a-1d0ee441b783 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:47.699396 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afb08f-1aab-417f-aeb4-67ce3d43783e None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:47.699493 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e000837-8bdc-4b48-9d5a-1d0ee441b783 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:47.699652 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afb08f-1aab-417f-aeb4-67ce3d43783e None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:47.700452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afb08f-1aab-417f-aeb4-67ce3d43783e None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:47.705835 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 234/460] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:47.712077 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e7878-6987-429c-9acd-ea87907fbb95 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:47.712358 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e7878-6987-429c-9acd-ea87907fbb95 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:47.712628 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e7878-6987-429c-9acd-ea87907fbb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:47.723449 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e000837-8bdc-4b48-9d5a-1d0ee441b783 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:47.724899 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e000837-8bdc-4b48-9d5a-1d0ee441b783 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:47.745522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-5b1e7878-6987-429c-9acd-ea87907fbb95 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:47.747291 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-5b1e7878-6987-429c-9acd-ea87907fbb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:47.751755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-2e000837-8bdc-4b48-9d5a-1d0ee441b783 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:47.751755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:47.751755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:47.751755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:47.751755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:47.751755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:47.751755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:47.751755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:47.751755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:47.751755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:47.751755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:47.751755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:47.751755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:59:47.751755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:59:47.751755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:59:47.751755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:59:47.752668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:47.752668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:47.752668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:47.752668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:47.752668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:47.752668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:47.752668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:47.752668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:47.752668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:47.752668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:47.752668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:47.752668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:47.752668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:47.752668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:47.752668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:47.752668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:47.752668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:59:47.753522 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:59:47.753522 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:47.753522 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:47.757732 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 227/461] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:47.764414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e000837-8bdc-4b48-9d5a-1d0ee441b783 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:47.836988 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:47.857018 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 235/462] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:59:47.857724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e7878-6987-429c-9acd-ea87907fbb95 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:47.868910 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:47.896959 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e000837-8bdc-4b48-9d5a-1d0ee441b783 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:47.932174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f345da1-8c3c-492a-ba40-73ce863ee6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:47.932889 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f345da1-8c3c-492a-ba40-73ce863ee6ab None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:47.933166 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f345da1-8c3c-492a-ba40-73ce863ee6ab None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:47.933433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f345da1-8c3c-492a-ba40-73ce863ee6ab None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:47.934832 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f345da1-8c3c-492a-ba40-73ce863ee6ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:47.935137 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f345da1-8c3c-492a-ba40-73ce863ee6ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:47.936643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f345da1-8c3c-492a-ba40-73ce863ee6ab None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:47.950534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 228/463] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 19:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:47.958404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f345da1-8c3c-492a-ba40-73ce863ee6ab None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:48.064844 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e7878-6987-429c-9acd-ea87907fbb95 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:48.119295 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0653a42-8bbe-4657-bd99-202a82e66206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:48.120715 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0653a42-8bbe-4657-bd99-202a82e66206 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:48.123009 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0653a42-8bbe-4657-bd99-202a82e66206 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:48.123501 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0653a42-8bbe-4657-bd99-202a82e66206 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:48.149434 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0653a42-8bbe-4657-bd99-202a82e66206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:48.157849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0653a42-8bbe-4657-bd99-202a82e66206 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:48.173143 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f345da1-8c3c-492a-ba40-73ce863ee6ab None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:48.176299 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:48.191672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-c0653a42-8bbe-4657-bd99-202a82e66206 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:48.191672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:48.191672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:48.191672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:48.191672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:48.191672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:48.191672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:48.191672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:48.191672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:48.191672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:48.191672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:48.191672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:48.191672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:59:48.191672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:59:48.191672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:59:48.191672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:59:48.192731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:48.192731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:48.192731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:48.192731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:48.192731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:48.192731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:48.192731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:48.192731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:48.192731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:48.192731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:48.192731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:48.192731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:48.192731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:48.192731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:48.192731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:48.192731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:48.192731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:59:48.193580 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:59:48.193580 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:48.193580 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:48.193580 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 236/464] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:48.194245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0653a42-8bbe-4657-bd99-202a82e66206 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:48.221687 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78cbce-dc62-424d-97ce-0f9db69f7b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:48.222695 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78cbce-dc62-424d-97ce-0f9db69f7b62 None admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:48.223196 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78cbce-dc62-424d-97ce-0f9db69f7b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:48.223708 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78cbce-dc62-424d-97ce-0f9db69f7b62 None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:48.224666 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78cbce-dc62-424d-97ce-0f9db69f7b62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:48.226365 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78cbce-dc62-424d-97ce-0f9db69f7b62 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:48.251623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 229/465] 10.222.0.73 () {60 vars in 1132 bytes} [Mon Aug 29 19:59:47 2022] POST /identity/v3/users => generated 246 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:59:48.253059 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78cbce-dc62-424d-97ce-0f9db69f7b62 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:48.268831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:48.356274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0653a42-8bbe-4657-bd99-202a82e66206 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:48.380896 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78cbce-dc62-424d-97ce-0f9db69f7b62 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:48.416339 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e133840-443c-4a2a-8695-b2df76d86925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:48.417335 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e133840-443c-4a2a-8695-b2df76d86925 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:48.417877 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e133840-443c-4a2a-8695-b2df76d86925 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:48.418360 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e133840-443c-4a2a-8695-b2df76d86925 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:48.427058 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a8f1c-7cc9-41fb-845d-c427d2fef54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:48.427974 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a8f1c-7cc9-41fb-845d-c427d2fef54a None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:48.428500 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a8f1c-7cc9-41fb-845d-c427d2fef54a None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:48.428971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a8f1c-7cc9-41fb-845d-c427d2fef54a None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:48.429915 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a8f1c-7cc9-41fb-845d-c427d2fef54a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:48.430471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a8f1c-7cc9-41fb-845d-c427d2fef54a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:48.432103 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a8f1c-7cc9-41fb-845d-c427d2fef54a None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:48.433372 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e133840-443c-4a2a-8695-b2df76d86925 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:48.435694 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e133840-443c-4a2a-8695-b2df76d86925 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:48.447473 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 230/466] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 19:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:48.449051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7912a-4451-42cd-8bf2-9989b997a95a None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:48.449516 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7912a-4451-42cd-8bf2-9989b997a95a None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:48.451099 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7912a-4451-42cd-8bf2-9989b997a95a None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:48.452659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 231/467] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:48.457642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-2e133840-443c-4a2a-8695-b2df76d86925 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:48.457642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:48.457642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:48.457642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:48.457642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:48.457642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:48.457642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:48.457642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:48.457642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:48.457642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:48.457642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:48.457642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:48.457642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:59:48.457642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:59:48.457642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:59:48.457642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:59:48.458576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:48.458576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:48.458576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:48.458576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:48.458576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:48.458576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:48.458576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:48.458576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:48.458576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:48.458576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:48.458576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:48.458576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:48.458576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:48.458576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:48.458576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:48.458576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:48.458576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:59:48.459432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:59:48.459432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:48.459432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:48.459592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c56631-a8a7-4e2d-aeca-b9a7365baaf0 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:48.459592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c56631-a8a7-4e2d-aeca-b9a7365baaf0 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:48.459592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c56631-a8a7-4e2d-aeca-b9a7365baaf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:48.461422 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 237/468] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:48.462422 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e133840-443c-4a2a-8695-b2df76d86925 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:48.484501 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-14c56631-a8a7-4e2d-aeca-b9a7365baaf0 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:48.485839 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-14c56631-a8a7-4e2d-aeca-b9a7365baaf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:48.507800 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:48.581103 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 232/469] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:48.581995 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c56631-a8a7-4e2d-aeca-b9a7365baaf0 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:48.607542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e133840-443c-4a2a-8695-b2df76d86925 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:48.627162 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:48.658856 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3df28-95c0-49d3-8c34-862295fb9f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:48.660354 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3df28-95c0-49d3-8c34-862295fb9f84 None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:48.660786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3df28-95c0-49d3-8c34-862295fb9f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:48.661510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3df28-95c0-49d3-8c34-862295fb9f84 None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:48.662643 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3df28-95c0-49d3-8c34-862295fb9f84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:48.665243 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3df28-95c0-49d3-8c34-862295fb9f84 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:48.696306 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 238/470] 10.222.0.73 () {60 vars in 1132 bytes} [Mon Aug 29 19:59:48 2022] POST /identity/v3/users => generated 248 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:59:48.697888 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b413006-c215-4443-94dd-e19cfbc83850 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:48.698065 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c56631-a8a7-4e2d-aeca-b9a7365baaf0 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:48.698479 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b413006-c215-4443-94dd-e19cfbc83850 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:48.698934 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b413006-c215-4443-94dd-e19cfbc83850 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:48.706275 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 239/471] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:48.709164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1079f2-00a6-4105-b998-eed9f61d08a0 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:48.709659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1079f2-00a6-4105-b998-eed9f61d08a0 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:48.710166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1079f2-00a6-4105-b998-eed9f61d08a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:48.736972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-6a1079f2-00a6-4105-b998-eed9f61d08a0 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:48.738560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-6a1079f2-00a6-4105-b998-eed9f61d08a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:48.750331 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20eb6d0-f044-4477-8dea-7e597070f21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:48.750998 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20eb6d0-f044-4477-8dea-7e597070f21e None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:48.751276 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20eb6d0-f044-4477-8dea-7e597070f21e None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:48.751545 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20eb6d0-f044-4477-8dea-7e597070f21e None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:48.752397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20eb6d0-f044-4477-8dea-7e597070f21e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:48.752569 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20eb6d0-f044-4477-8dea-7e597070f21e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:48.754186 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20eb6d0-f044-4477-8dea-7e597070f21e None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:48.777420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 233/472] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 19:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:48.784004 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20eb6d0-f044-4477-8dea-7e597070f21e None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:48.814172 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:48.846084 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 240/473] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 19:59:48.884810 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1079f2-00a6-4105-b998-eed9f61d08a0 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:48.981249 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20eb6d0-f044-4477-8dea-7e597070f21e None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:49.024394 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81589dcf-7d8d-4d2e-a885-ffdaccd22a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:49.025399 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81589dcf-7d8d-4d2e-a885-ffdaccd22a97 None admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:49.025901 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81589dcf-7d8d-4d2e-a885-ffdaccd22a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:49.026533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81589dcf-7d8d-4d2e-a885-ffdaccd22a97 None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:49.027484 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81589dcf-7d8d-4d2e-a885-ffdaccd22a97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:49.029144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81589dcf-7d8d-4d2e-a885-ffdaccd22a97 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:49.029636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:49.033891 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1079f2-00a6-4105-b998-eed9f61d08a0 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:49.060341 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 234/474] 10.222.0.73 () {60 vars in 1132 bytes} [Mon Aug 29 19:59:48 2022] POST /identity/v3/users => generated 248 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:59:49.104108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa675087-aa43-4668-8d44-6ef3496519fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:49.104834 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa675087-aa43-4668-8d44-6ef3496519fb None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:49.105108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa675087-aa43-4668-8d44-6ef3496519fb None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:49.105373 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa675087-aa43-4668-8d44-6ef3496519fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:49.117966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa675087-aa43-4668-8d44-6ef3496519fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:49.119443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa675087-aa43-4668-8d44-6ef3496519fb None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:49.137050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-aa675087-aa43-4668-8d44-6ef3496519fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:49.137050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:49.137050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:49.137050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:49.137050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:49.137050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:49.137050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:49.137050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:49.137050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:49.137050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:49.137050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:49.137050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:49.137050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:59:49.137050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:59:49.137050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:59:49.137050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:59:49.138766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:49.138766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:49.138766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:49.138766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:49.138766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:49.138766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:49.138766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:49.138766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:49.138766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:49.138766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:49.138766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:49.138766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:49.138766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:49.138766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:49.138766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:49.138766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:49.138766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:59:49.139610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:59:49.139610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:49.139610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:49.139610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 241/475] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:49.144995 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81589dcf-7d8d-4d2e-a885-ffdaccd22a97 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:49.154802 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa675087-aa43-4668-8d44-6ef3496519fb None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:49.261107 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:49.290048 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa675087-aa43-4668-8d44-6ef3496519fb None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:49.303644 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81589dcf-7d8d-4d2e-a885-ffdaccd22a97 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:49.329353 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df2029-4e29-441c-a704-15a3e7b61520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:49.330074 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df2029-4e29-441c-a704-15a3e7b61520 None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:49.330355 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df2029-4e29-441c-a704-15a3e7b61520 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:49.330628 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df2029-4e29-441c-a704-15a3e7b61520 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:49.331309 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df2029-4e29-441c-a704-15a3e7b61520 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:49.332891 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df2029-4e29-441c-a704-15a3e7b61520 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:49.342610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.sql.core [None req-f2df2029-4e29-441c-a704-15a3e7b61520 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 19:59:49.342610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 29 19:59:49.342610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [parameters: {'id': '8a6ba556df194f73a58ab5a6f46efe66', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 29 19:59:49.342610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81782) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 19:59:49.346732 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b247d25-31db-4a8e-808a-183670cd0822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:49.347450 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b247d25-31db-4a8e-808a-183670cd0822 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:49.347849 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b247d25-31db-4a8e-808a-183670cd0822 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:49.348312 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b247d25-31db-4a8e-808a-183670cd0822 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:49.349114 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b247d25-31db-4a8e-808a-183670cd0822 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:49.349418 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b247d25-31db-4a8e-808a-183670cd0822 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:49.350940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b247d25-31db-4a8e-808a-183670cd0822 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:49.365416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-f2df2029-4e29-441c-a704-15a3e7b61520 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 29 19:59:49.365416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:49.365416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 19:59:49.365416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 19:59:49.365416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 19:59:49.365416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 19:59:49.365416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 19:59:49.365416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 19:59:49.365416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 19:59:49.365416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application conn.query(q) Aug 29 19:59:49.365416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 19:59:49.365416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 19:59:49.365416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 19:59:49.365416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application result.read() Aug 29 19:59:49.365416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 19:59:49.366582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 19:59:49.366582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 19:59:49.366582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 19:59:49.366582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 19:59:49.366582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 19:59:49.366582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 19:59:49.366582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 19:59:49.366582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 19:59:49.366582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:49.366582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 19:59:49.366582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:49.366582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:49.366582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 19:59:49.366582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 19:59:49.366582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 29 19:59:49.366582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return ref.to_dict() Aug 29 19:59:49.366582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 19:59:49.366582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application next(self.gen) Aug 29 19:59:49.367771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 19:59:49.367771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application yield resource Aug 29 19:59:49.367771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 19:59:49.367771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application next(self.gen) Aug 29 19:59:49.367771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 19:59:49.367771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 19:59:49.367771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 19:59:49.367771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 19:59:49.367771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 19:59:49.367771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise self.value Aug 29 19:59:49.367771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 19:59:49.367771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 19:59:49.367771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 19:59:49.367771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application session.commit() Aug 29 19:59:49.367771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 19:59:49.367771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 19:59:49.367771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 19:59:49.368895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 19:59:49.368895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 19:59:49.368895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self.session.flush() Aug 29 19:59:49.368895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 19:59:49.368895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 19:59:49.368895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 19:59:49.368895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 19:59:49.368895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 19:59:49.368895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application compat.raise_( Aug 29 19:59:49.368895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 19:59:49.368895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception Aug 29 19:59:49.368895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 19:59:49.368895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 19:59:49.368895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 19:59:49.368895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 19:59:49.368895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 19:59:49.368895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 19:59:49.369984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 19:59:49.369984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 19:59:49.369984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 29 19:59:49.369984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 29 19:59:49.369984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 19:59:49.369984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 19:59:49.369984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 19:59:49.369984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 19:59:49.369984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 19:59:49.369984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 19:59:49.369984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 19:59:49.369984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 19:59:49.369984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 19:59:49.369984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 19:59:49.369984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 19:59:49.369984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception Aug 29 19:59:49.370989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 19:59:49.370989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 19:59:49.370989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 19:59:49.370989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 19:59:49.370989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 19:59:49.370989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 19:59:49.370989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 19:59:49.370989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application conn.query(q) Aug 29 19:59:49.370989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 19:59:49.370989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 19:59:49.370989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 19:59:49.370989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application result.read() Aug 29 19:59:49.370989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 19:59:49.370989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 19:59:49.370989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 19:59:49.370989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 19:59:49.377880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 19:59:49.377880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 19:59:49.377880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 19:59:49.377880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 19:59:49.377880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 19:59:49.377880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 29 19:59:49.377880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [parameters: {'id': '8a6ba556df194f73a58ab5a6f46efe66', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 29 19:59:49.377880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 19:59:49.377880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:49.377880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 19:59:49.377880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:49.377880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:49.377880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:49.377880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:49.377880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:49.377880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:49.379496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:49.379496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:49.379496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:49.379496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:49.379496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:49.379496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:49.379496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 29 19:59:49.379496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 29 19:59:49.379496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:49.379496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:49.379496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 29 19:59:49.379496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 29 19:59:49.379496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 19:59:49.379496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 19:59:49.379496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 29 19:59:49.379496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:49.380721 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 235/476] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 19:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:49.380850 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 242/477] 10.222.0.73 () {60 vars in 1131 bytes} [Mon Aug 29 19:59:49 2022] POST /identity/v3/roles => generated 144 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 29 19:59:49.380972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b247d25-31db-4a8e-808a-183670cd0822 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:49.389550 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df2029-4e29-441c-a704-15a3e7b61520 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:49.446115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:49.508791 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b247d25-31db-4a8e-808a-183670cd0822 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:49.537868 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:49.544072 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df2029-4e29-441c-a704-15a3e7b61520 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:49.557307 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbd8a7-9684-437f-95b5-591efc5abf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:49.558340 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbd8a7-9684-437f-95b5-591efc5abf97 None admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:49.558828 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbd8a7-9684-437f-95b5-591efc5abf97 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:49.560176 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbd8a7-9684-437f-95b5-591efc5abf97 None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:49.561063 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dbd8a7-9684-437f-95b5-591efc5abf97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:49.562743 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dbd8a7-9684-437f-95b5-591efc5abf97 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:49.584857 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb1da2-8784-4116-96bd-9dbf7b03d94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:49.585510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb1da2-8784-4116-96bd-9dbf7b03d94b None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:49.585926 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb1da2-8784-4116-96bd-9dbf7b03d94b None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:49.586182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb1da2-8784-4116-96bd-9dbf7b03d94b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:49.597240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 236/478] 10.222.0.73 () {60 vars in 1132 bytes} [Mon Aug 29 19:59:49 2022] POST /identity/v3/users => generated 249 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:59:49.602167 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb1da2-8784-4116-96bd-9dbf7b03d94b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:49.612200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-0cbb1da2-8784-4116-96bd-9dbf7b03d94b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:49.612200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:49.612200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:49.612200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:49.612200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:49.612200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:49.612200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:49.612200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:49.612200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:49.612200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:49.612200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:49.612200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:49.612200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:49.612200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:49.612200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:49.612200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:49.613155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:49.613155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:49.613155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:49.613155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:49.613155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:49.613155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:49.613155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:49.613155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:49.613155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:49.613155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:49.613155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:49.613155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:49.613155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:49.613155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:49.613155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:49.613155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:49.614402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:49.614402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:49.614402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:49.614402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:49.614402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:49.614402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:49.614402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:49.614402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:49.614402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:49.614402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:49.614402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 19:59:49.614402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:49.614402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 243/479] 10.222.0.73 () {56 vars in 1114 bytes} [Mon Aug 29 19:59:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 19:59:49.621337 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbd8a7-9684-437f-95b5-591efc5abf97 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:49.754130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbd8a7-9684-437f-95b5-591efc5abf97 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:49.789442 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3763ec9-20be-4387-9908-c3bfaff97301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:49.790435 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3763ec9-20be-4387-9908-c3bfaff97301 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:49.790915 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3763ec9-20be-4387-9908-c3bfaff97301 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:49.791402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3763ec9-20be-4387-9908-c3bfaff97301 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:49.792368 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3763ec9-20be-4387-9908-c3bfaff97301 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:49.792874 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3763ec9-20be-4387-9908-c3bfaff97301 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:49.795181 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3763ec9-20be-4387-9908-c3bfaff97301 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:49.809853 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 237/480] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 19:59:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:50.562722 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced49e2-af77-4be1-bfdd-795aeb572096 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:50.563204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced49e2-af77-4be1-bfdd-795aeb572096 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:50.563204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced49e2-af77-4be1-bfdd-795aeb572096 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:50.564679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 244/481] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:50.570200 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b97e2-eb2e-41a0-aaff-29dba10a1565 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:50.570628 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b97e2-eb2e-41a0-aaff-29dba10a1565 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:50.570821 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b97e2-eb2e-41a0-aaff-29dba10a1565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:50.598761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-1c5b97e2-eb2e-41a0-aaff-29dba10a1565 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:50.600008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-1c5b97e2-eb2e-41a0-aaff-29dba10a1565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:50.666096 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:50.704392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 238/482] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:51.075031 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced49e2-af77-4be1-bfdd-795aeb572096 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:51.175833 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:51.206196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced49e2-af77-4be1-bfdd-795aeb572096 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:51.242937 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86418e93-93e7-4947-aae2-aaff05210e56 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:51.243284 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86418e93-93e7-4947-aae2-aaff05210e56 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:51.243479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86418e93-93e7-4947-aae2-aaff05210e56 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:51.245358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 239/483] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:51.246586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d52337-eba0-4366-97f2-71a9946b6d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:51.248050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d52337-eba0-4366-97f2-71a9946b6d07 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:51.248383 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d52337-eba0-4366-97f2-71a9946b6d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:51.248657 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d52337-eba0-4366-97f2-71a9946b6d07 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:51.252729 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a54432-da6e-4499-9269-82a81f0b05f4 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:51.253260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a54432-da6e-4499-9269-82a81f0b05f4 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:51.253811 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a54432-da6e-4499-9269-82a81f0b05f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:51.260329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d52337-eba0-4366-97f2-71a9946b6d07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:51.266294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-e9d52337-eba0-4366-97f2-71a9946b6d07 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 19:59:51.266294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:51.266294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:51.266294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:51.266294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:51.266294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:51.266294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:51.266294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:51.266294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:51.266294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:51.266294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:51.266294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:51.266294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:51.266294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:51.266294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:51.266294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:51.267164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:51.267164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:51.267164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:51.267164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:51.267164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:51.267164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:51.267164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:51.267164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:51.267164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:51.267164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:51.267164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:51.267164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:51.267164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:51.267164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:51.267164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:51.267164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:51.267975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:51.267975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:51.267975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:51.267975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:51.267975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:51.267975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:51.267975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:51.267975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:51.267975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:51.267975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:51.267975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 19:59:51.267975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:51.267975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 245/484] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 19:59:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:51.273633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d52337-eba0-4366-97f2-71a9946b6d07 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:51.298772 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-34a54432-da6e-4499-9269-82a81f0b05f4 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:51.300304 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-34a54432-da6e-4499-9269-82a81f0b05f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:51.414955 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 240/485] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:51.441533 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d52337-eba0-4366-97f2-71a9946b6d07 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:51.499755 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfaf49c-9ccd-4542-9644-864468d80438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:51.500371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfaf49c-9ccd-4542-9644-864468d80438 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:51.500651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfaf49c-9ccd-4542-9644-864468d80438 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:51.500925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfaf49c-9ccd-4542-9644-864468d80438 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:51.501590 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfaf49c-9ccd-4542-9644-864468d80438 None admin] RBAC: Adding query filter params (name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:51.501969 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfaf49c-9ccd-4542-9644-864468d80438 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:51.503456 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfaf49c-9ccd-4542-9644-864468d80438 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:51.505066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:51.517452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 246/486] 10.222.0.73 () {56 vars in 1103 bytes} [Mon Aug 29 19:59:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:51.522982 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a54432-da6e-4499-9269-82a81f0b05f4 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:51.667115 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a54432-da6e-4499-9269-82a81f0b05f4 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:51.671089 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:51.683712 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202da01f-4572-48e7-aba9-ac0855292b0e None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:51.685194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202da01f-4572-48e7-aba9-ac0855292b0e None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:51.685646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202da01f-4572-48e7-aba9-ac0855292b0e None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:51.693807 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 247/487] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:59:51.698989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227db6c-5ba9-453b-bc17-29a66925301b None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:51.699484 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227db6c-5ba9-453b-bc17-29a66925301b None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:51.699994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227db6c-5ba9-453b-bc17-29a66925301b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:51.709131 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc97e68-537b-4048-96a6-cc73101b6020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:51.709794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc97e68-537b-4048-96a6-cc73101b6020 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:51.710070 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc97e68-537b-4048-96a6-cc73101b6020 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:51.710338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc97e68-537b-4048-96a6-cc73101b6020 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:51.723088 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc97e68-537b-4048-96a6-cc73101b6020 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:51.724535 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc97e68-537b-4048-96a6-cc73101b6020 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:51.741757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-dfc97e68-537b-4048-96a6-cc73101b6020 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:51.741757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:51.741757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:51.741757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:51.741757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:51.741757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:51.741757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:51.741757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:51.741757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:51.741757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:51.741757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:51.741757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:51.741757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:59:51.741757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:59:51.741757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:59:51.741757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:59:51.742688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:51.742688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:51.742688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:51.742688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:51.742688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:51.742688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:51.742688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:51.742688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:51.742688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:51.742688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:51.742688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:51.742688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:51.742688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:51.742688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:51.742688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:51.742688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:51.742688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:59:51.743792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:59:51.743792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:51.743792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:51.743792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 241/488] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:51.744174 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-8227db6c-5ba9-453b-bc17-29a66925301b None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:51.745402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-8227db6c-5ba9-453b-bc17-29a66925301b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:51.755119 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc97e68-537b-4048-96a6-cc73101b6020 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:51.840006 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 248/489] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:51.841647 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227db6c-5ba9-453b-bc17-29a66925301b None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:51.902144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc97e68-537b-4048-96a6-cc73101b6020 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:51.960841 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227db6c-5ba9-453b-bc17-29a66925301b None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:51.979300 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e5253-8d11-4817-bc6b-6219a9e9698b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:51.980106 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e5253-8d11-4817-bc6b-6219a9e9698b None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:51.980215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e5253-8d11-4817-bc6b-6219a9e9698b None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:51.980445 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e5253-8d11-4817-bc6b-6219a9e9698b None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:51.981157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e5253-8d11-4817-bc6b-6219a9e9698b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:51.981507 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e5253-8d11-4817-bc6b-6219a9e9698b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:51.983057 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e5253-8d11-4817-bc6b-6219a9e9698b None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:51.994061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:52.006731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c97a8c-83cd-4e64-8126-391e943198ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:52.007711 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c97a8c-83cd-4e64-8126-391e943198ca None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:52.008178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c97a8c-83cd-4e64-8126-391e943198ca None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:52.008645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c97a8c-83cd-4e64-8126-391e943198ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:52.012227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 242/490] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 19:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:52.018819 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e5253-8d11-4817-bc6b-6219a9e9698b None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:52.026848 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c97a8c-83cd-4e64-8126-391e943198ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:52.033140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-e4c97a8c-83cd-4e64-8126-391e943198ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 19:59:52.033140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:52.033140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:52.033140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:52.033140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:52.033140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:52.033140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:52.033140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:52.033140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:52.033140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:52.033140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:52.033140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:52.033140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:52.033140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:52.033140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:52.033140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:52.034082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:52.034082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:52.034082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:52.034082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:52.034082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:52.034082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:52.034082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:52.034082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:52.034082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:52.034082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:52.034082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:52.034082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:52.034082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:52.034082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:52.034082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:52.034082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:52.034866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:52.034866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:52.034866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:52.034866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:52.034866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:52.034866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:52.034866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:52.034866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:52.034866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:52.034866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:52.034866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 19:59:52.034866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:52.037464 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 249/491] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 19:59:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:52.043424 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c97a8c-83cd-4e64-8126-391e943198ca None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:52.067208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:52.187378 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e5253-8d11-4817-bc6b-6219a9e9698b None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:52.227563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde5ca4-96d4-4f24-bd57-bfe46bb12afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:52.227563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde5ca4-96d4-4f24-bd57-bfe46bb12afa None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:52.227563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde5ca4-96d4-4f24-bd57-bfe46bb12afa None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:52.227563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde5ca4-96d4-4f24-bd57-bfe46bb12afa None admin] PATH_INFO: `/v3/users/nova` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:52.230529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c97a8c-83cd-4e64-8126-391e943198ca None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:52.237649 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde5ca4-96d4-4f24-bd57-bfe46bb12afa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:52.237649 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde5ca4-96d4-4f24-bd57-bfe46bb12afa None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:52.252416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-7bde5ca4-96d4-4f24-bd57-bfe46bb12afa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 19:59:52.252416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:52.252416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:52.252416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:52.252416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:52.252416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:52.252416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:52.252416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:52.252416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:52.252416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:52.252416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:52.252416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:52.252416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 19:59:52.252416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 19:59:52.252416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 19:59:52.252416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 19:59:52.253379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:52.253379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:52.253379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 19:59:52.253379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:59:52.253379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 19:59:52.253379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:59:52.253379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:52.253379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:52.253379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:52.253379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:52.253379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:52.253379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:52.253379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:52.253379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:52.253379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:52.253379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:52.253379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:52.253379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:52.254282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:52.254282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:52.254282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 19:59:52.254282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 19:59:52.254282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 19:59:52.254282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 19:59:52.254282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 19:59:52.254282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 19:59:52.254282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 19:59:52.254282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:52.258164 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 243/492] 10.222.0.73 () {56 vars in 1093 bytes} [Mon Aug 29 19:59:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 19:59:52.260029 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde5ca4-96d4-4f24-bd57-bfe46bb12afa None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:52.297986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7fa88-ea5d-49cf-8b9c-46610d57c2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:52.298593 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7fa88-ea5d-49cf-8b9c-46610d57c2e1 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:52.298869 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7fa88-ea5d-49cf-8b9c-46610d57c2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:52.299126 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7fa88-ea5d-49cf-8b9c-46610d57c2e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:52.299785 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7fa88-ea5d-49cf-8b9c-46610d57c2e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:52.300083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7fa88-ea5d-49cf-8b9c-46610d57c2e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:52.301455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7fa88-ea5d-49cf-8b9c-46610d57c2e1 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:52.310315 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 250/493] 10.222.0.73 () {56 vars in 1103 bytes} [Mon Aug 29 19:59:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:52.314928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf84d0-43a2-4486-b387-3c32ce5d6539 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:52.315211 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf84d0-43a2-4486-b387-3c32ce5d6539 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:52.315487 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf84d0-43a2-4486-b387-3c32ce5d6539 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:52.316769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 251/494] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:52.317465 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf84d0-43a2-4486-b387-3c32ce5d6539 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:52.349659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:52.371241 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:52.374705 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde5ca4-96d4-4f24-bd57-bfe46bb12afa None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:52.410271 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f251cd-a7ed-4967-a7ad-9e78d308f8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:52.410870 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f251cd-a7ed-4967-a7ad-9e78d308f8c7 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:52.411148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f251cd-a7ed-4967-a7ad-9e78d308f8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:52.411421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f251cd-a7ed-4967-a7ad-9e78d308f8c7 None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:52.412146 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f251cd-a7ed-4967-a7ad-9e78d308f8c7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:52.412452 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f251cd-a7ed-4967-a7ad-9e78d308f8c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:52.413925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f251cd-a7ed-4967-a7ad-9e78d308f8c7 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:52.425950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf84d0-43a2-4486-b387-3c32ce5d6539 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:52.451440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 244/495] 10.222.0.73 () {56 vars in 1133 bytes} [Mon Aug 29 19:59:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:52.451440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af2d4c-a8a1-4a9d-9b1e-22b6d839b065 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:52.451440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af2d4c-a8a1-4a9d-9b1e-22b6d839b065 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:52.451440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af2d4c-a8a1-4a9d-9b1e-22b6d839b065 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:52.451440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 245/496] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:52.451440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af2d4c-a8a1-4a9d-9b1e-22b6d839b065 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:52.466780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b835cc65-5827-46b8-b4cf-76f50db5d8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:52.467421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835cc65-5827-46b8-b4cf-76f50db5d8c9 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:52.467697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835cc65-5827-46b8-b4cf-76f50db5d8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:52.467962 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835cc65-5827-46b8-b4cf-76f50db5d8c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:52.478248 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b835cc65-5827-46b8-b4cf-76f50db5d8c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:52.483117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-b835cc65-5827-46b8-b4cf-76f50db5d8c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 19:59:52.483117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:52.483117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:52.483117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:52.483117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:52.483117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:52.483117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:52.483117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:52.483117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:52.483117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:52.483117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:52.483117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:52.483117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:52.483117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:52.483117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:52.483117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:52.483986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:52.483986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:52.483986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:52.483986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:52.483986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:52.483986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:52.483986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:52.483986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:52.483986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:52.483986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:52.483986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:52.483986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:52.483986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:52.483986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:52.483986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:52.483986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:52.484814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:52.484814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:52.484814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:52.484814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:52.484814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:52.484814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:52.484814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:52.484814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:52.484814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:52.484814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:52.484814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 19:59:52.484814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:52.484814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 252/497] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 19:59:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:52.485456 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196138d-4ac1-4315-a209-19f66ced776c None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:52.485620 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196138d-4ac1-4315-a209-19f66ced776c None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:52.485908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196138d-4ac1-4315-a209-19f66ced776c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:52.504875 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-2196138d-4ac1-4315-a209-19f66ced776c None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:52.506111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-2196138d-4ac1-4315-a209-19f66ced776c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:52.565083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af2d4c-a8a1-4a9d-9b1e-22b6d839b065 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:52.572469 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 253/498] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:52.573314 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196138d-4ac1-4315-a209-19f66ced776c None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:52.601309 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6341801-62dc-405d-86cf-77468985a943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:52.601965 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6341801-62dc-405d-86cf-77468985a943 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:52.602353 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6341801-62dc-405d-86cf-77468985a943 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:52.602509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6341801-62dc-405d-86cf-77468985a943 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:52.604514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:52.614575 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6341801-62dc-405d-86cf-77468985a943 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:52.616046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6341801-62dc-405d-86cf-77468985a943 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:52.617150 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:52.634722 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-c6341801-62dc-405d-86cf-77468985a943 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:52.634722 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:52.634722 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:52.634722 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:52.634722 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:52.634722 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:52.634722 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:52.634722 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:52.634722 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:52.634722 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:52.634722 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:52.634722 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:52.634722 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:59:52.634722 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:59:52.634722 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:59:52.634722 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:59:52.635692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:52.635692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:52.635692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:52.635692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:52.635692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:52.635692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:52.635692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:52.635692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:52.635692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:52.635692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:52.635692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:52.635692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:52.635692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:52.635692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:52.635692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:52.635692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:52.635692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:59:52.636528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:59:52.636528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:52.636528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:52.636528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 246/499] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:52.637400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba13ec-b0e5-4425-8f38-ffdeac3e1a7e None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:52.637400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba13ec-b0e5-4425-8f38-ffdeac3e1a7e None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:52.638276 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba13ec-b0e5-4425-8f38-ffdeac3e1a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:52.658444 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-9bba13ec-b0e5-4425-8f38-ffdeac3e1a7e None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:52.659635 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9bba13ec-b0e5-4425-8f38-ffdeac3e1a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:52.693782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196138d-4ac1-4315-a209-19f66ced776c None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:52.731864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b98b5-ffcf-43a2-a993-a773e8cd05fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:52.732210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b98b5-ffcf-43a2-a993-a773e8cd05fd None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:52.732604 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b98b5-ffcf-43a2-a993-a773e8cd05fd None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:52.732880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b98b5-ffcf-43a2-a993-a773e8cd05fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:52.749730 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 247/500] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:52.749730 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba13ec-b0e5-4425-8f38-ffdeac3e1a7e None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:52.814915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b98b5-ffcf-43a2-a993-a773e8cd05fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:52.816518 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b98b5-ffcf-43a2-a993-a773e8cd05fd None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:52.859640 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:52.873302 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba13ec-b0e5-4425-8f38-ffdeac3e1a7e None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:52.892333 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:52.912436 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab77db-a49e-4865-9095-91bb65a72633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:52.913141 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab77db-a49e-4865-9095-91bb65a72633 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:52.913415 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab77db-a49e-4865-9095-91bb65a72633 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:52.913683 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab77db-a49e-4865-9095-91bb65a72633 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:52.914363 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab77db-a49e-4865-9095-91bb65a72633 None admin] RBAC: Adding query filter params (name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:52.914661 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab77db-a49e-4865-9095-91bb65a72633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:52.916229 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab77db-a49e-4865-9095-91bb65a72633 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:52.925756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 248/501] 10.222.0.73 () {56 vars in 1103 bytes} [Mon Aug 29 19:59:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:52.926463 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab77db-a49e-4865-9095-91bb65a72633 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:52.937925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 254/502] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:52.937925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b98b5-ffcf-43a2-a993-a773e8cd05fd None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:53.037046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab77db-a49e-4865-9095-91bb65a72633 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:53.047483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b98b5-ffcf-43a2-a993-a773e8cd05fd None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:53.071563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b2486-929f-431f-931d-5617addd487d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:53.072173 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b2486-929f-431f-931d-5617addd487d None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:53.072456 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b2486-929f-431f-931d-5617addd487d None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:53.072724 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b2486-929f-431f-931d-5617addd487d None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:53.073360 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b2486-929f-431f-931d-5617addd487d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:53.073660 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b2486-929f-431f-931d-5617addd487d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:53.075584 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b2486-929f-431f-931d-5617addd487d None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:53.081290 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0dc6e9-9075-4084-a39a-09dbf7711f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:53.081950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dc6e9-9075-4084-a39a-09dbf7711f98 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:53.082230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dc6e9-9075-4084-a39a-09dbf7711f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:53.082504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dc6e9-9075-4084-a39a-09dbf7711f98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:53.088496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 249/503] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 19:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:53.089226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b2486-929f-431f-931d-5617addd487d None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:53.090264 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:53.095803 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0dc6e9-9075-4084-a39a-09dbf7711f98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:53.097596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0dc6e9-9075-4084-a39a-09dbf7711f98 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:53.112845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-6d0dc6e9-9075-4084-a39a-09dbf7711f98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:53.112845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:53.112845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:53.112845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:53.112845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:53.112845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:53.112845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:53.112845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:53.112845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:53.112845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:53.112845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:53.112845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:53.112845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:59:53.112845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:59:53.112845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:59:53.112845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:59:53.113811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:53.113811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:53.113811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:53.113811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:53.113811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:53.113811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:53.113811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:53.113811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:53.113811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:53.113811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:53.113811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:53.113811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:53.113811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:53.113811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:53.113811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:53.113811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:53.113811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:59:53.114665 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:59:53.114665 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:53.114665 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:53.114665 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 255/504] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:53.115046 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0dc6e9-9075-4084-a39a-09dbf7711f98 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:53.132946 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:53.195861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b2486-929f-431f-931d-5617addd487d None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:53.217086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0dc6e9-9075-4084-a39a-09dbf7711f98 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:53.230171 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f234a980-e419-4bb5-9348-f8121ab14b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:53.230802 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234a980-e419-4bb5-9348-f8121ab14b79 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:53.231617 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234a980-e419-4bb5-9348-f8121ab14b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:53.231896 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234a980-e419-4bb5-9348-f8121ab14b79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:53.242268 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f234a980-e419-4bb5-9348-f8121ab14b79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:53.243708 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f234a980-e419-4bb5-9348-f8121ab14b79 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:53.252230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8662b6-dcfb-45ab-9c2d-3ef642dfafaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:53.252859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8662b6-dcfb-45ab-9c2d-3ef642dfafaa None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:53.253143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8662b6-dcfb-45ab-9c2d-3ef642dfafaa None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:53.253419 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8662b6-dcfb-45ab-9c2d-3ef642dfafaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:53.259358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-f234a980-e419-4bb5-9348-f8121ab14b79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:53.259358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:53.259358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:53.259358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:53.259358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:53.259358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:53.259358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:53.259358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:53.259358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:53.259358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:53.259358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:53.259358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:53.259358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:59:53.259358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:59:53.259358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:59:53.259358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:59:53.260425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:53.260425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:53.260425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:53.260425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:53.260425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:53.260425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:53.260425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:53.260425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:53.260425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:53.260425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:53.260425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:53.260425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:53.260425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:53.260425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:53.260425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:53.260425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:53.260425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:59:53.261401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:59:53.261401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:53.261401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:53.261401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 250/505] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:53.261401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f234a980-e419-4bb5-9348-f8121ab14b79 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:53.266076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8662b6-dcfb-45ab-9c2d-3ef642dfafaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:53.267521 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8662b6-dcfb-45ab-9c2d-3ef642dfafaa None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:53.282537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-6f8662b6-dcfb-45ab-9c2d-3ef642dfafaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:53.282537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:53.282537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:53.282537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:53.282537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:53.282537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:53.282537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:53.282537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:53.282537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:53.282537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:53.282537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:53.282537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:53.282537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:59:53.282537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:59:53.282537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:59:53.282537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:59:53.283646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:53.283646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:53.283646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:53.283646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:53.283646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:53.283646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:53.283646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:53.283646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:53.283646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:53.283646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:53.283646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:53.283646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:53.283646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:53.283646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:53.283646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:53.283646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:53.283646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:59:53.284495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:59:53.284495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:53.284495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:53.284495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 256/506] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:53.284823 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8662b6-dcfb-45ab-9c2d-3ef642dfafaa None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:53.328113 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:53.357543 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:53.368074 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f234a980-e419-4bb5-9348-f8121ab14b79 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:53.387333 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8662b6-dcfb-45ab-9c2d-3ef642dfafaa None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:53.402011 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ff988-ee06-4406-9196-3530d5318abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:53.402636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ff988-ee06-4406-9196-3530d5318abe None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:53.402912 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ff988-ee06-4406-9196-3530d5318abe None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:53.403720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ff988-ee06-4406-9196-3530d5318abe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:53.413579 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ff988-ee06-4406-9196-3530d5318abe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:53.419319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-948ff988-ee06-4406-9196-3530d5318abe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 19:59:53.419319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:53.419319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:53.419319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:53.419319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:53.419319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:53.419319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:53.419319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:53.419319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:53.419319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:53.419319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:53.419319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:53.419319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:53.419319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:53.419319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:53.419319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:53.420226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:53.420226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:53.420226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:53.420226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:53.420226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:53.420226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:53.420226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:53.420226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:53.420226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:53.420226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:53.420226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:53.420226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:53.420226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:53.420226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:53.420226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:53.420226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:53.421055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:53.421055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:53.421055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:53.421055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:53.421055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:53.421055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:53.421055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:53.421055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:53.421055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:53.421055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:53.421055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 19:59:53.421055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:53.421055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 251/507] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 19:59:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:53.421727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ff988-ee06-4406-9196-3530d5318abe None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:53.426136 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331fc953-1d32-4e5b-8ff5-efd8560ef74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:53.426771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331fc953-1d32-4e5b-8ff5-efd8560ef74c None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:53.427048 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331fc953-1d32-4e5b-8ff5-efd8560ef74c None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:53.427316 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331fc953-1d32-4e5b-8ff5-efd8560ef74c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:53.435072 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331fc953-1d32-4e5b-8ff5-efd8560ef74c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:53.436526 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331fc953-1d32-4e5b-8ff5-efd8560ef74c None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:53.448942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-331fc953-1d32-4e5b-8ff5-efd8560ef74c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 19:59:53.448942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:53.448942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:53.448942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:53.448942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:53.448942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:53.448942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:53.448942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:53.448942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:53.448942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:53.448942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:53.448942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:53.448942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 19:59:53.448942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 19:59:53.448942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 19:59:53.448942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 19:59:53.450378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:53.450378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:53.450378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 19:59:53.450378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:59:53.450378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 19:59:53.450378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:59:53.450378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:53.450378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:53.450378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:53.450378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:53.450378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:53.450378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:53.450378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:53.450378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:53.450378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:53.450378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:53.450378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:53.450378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:53.451297 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:53.451297 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:53.451297 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 19:59:53.451297 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 19:59:53.451297 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 19:59:53.451297 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 19:59:53.451297 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 19:59:53.451297 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 19:59:53.451297 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 19:59:53.451297 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:53.451297 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 257/508] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:53.451297 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331fc953-1d32-4e5b-8ff5-efd8560ef74c None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:53.527097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ff988-ee06-4406-9196-3530d5318abe None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:53.553388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331fc953-1d32-4e5b-8ff5-efd8560ef74c None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:53.554976 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:53.563306 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afc4e1-51bc-4902-b5a1-f2409c5f6339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:53.563995 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afc4e1-51bc-4902-b5a1-f2409c5f6339 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:53.564271 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afc4e1-51bc-4902-b5a1-f2409c5f6339 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:53.564539 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afc4e1-51bc-4902-b5a1-f2409c5f6339 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:53.565174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6afc4e1-51bc-4902-b5a1-f2409c5f6339 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:53.565478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6afc4e1-51bc-4902-b5a1-f2409c5f6339 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:53.566876 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6afc4e1-51bc-4902-b5a1-f2409c5f6339 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:53.578602 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 252/509] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 19:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:53.579395 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afc4e1-51bc-4902-b5a1-f2409c5f6339 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:53.589393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074cdfd-428a-4e25-b9ed-56bd211d1c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:53.590024 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074cdfd-428a-4e25-b9ed-56bd211d1c18 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:53.590302 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074cdfd-428a-4e25-b9ed-56bd211d1c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:53.591164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074cdfd-428a-4e25-b9ed-56bd211d1c18 None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:53.592225 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074cdfd-428a-4e25-b9ed-56bd211d1c18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:53.592532 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074cdfd-428a-4e25-b9ed-56bd211d1c18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:53.594314 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074cdfd-428a-4e25-b9ed-56bd211d1c18 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:53.595200 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:53.607971 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 258/510] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 19:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:53.608881 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074cdfd-428a-4e25-b9ed-56bd211d1c18 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:53.688630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afc4e1-51bc-4902-b5a1-f2409c5f6339 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:53.711317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074cdfd-428a-4e25-b9ed-56bd211d1c18 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:53.724799 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f4b33-bf25-4602-8366-065d512a1365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:53.725397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f4b33-bf25-4602-8366-065d512a1365 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:53.725672 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f4b33-bf25-4602-8366-065d512a1365 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:53.725976 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f4b33-bf25-4602-8366-065d512a1365 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:53.726609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f4b33-bf25-4602-8366-065d512a1365 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:53.726906 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f4b33-bf25-4602-8366-065d512a1365 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:53.728272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f4b33-bf25-4602-8366-065d512a1365 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:53.739737 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 253/511] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 19:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:53.740573 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f4b33-bf25-4602-8366-065d512a1365 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:53.748344 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4e986-d5d1-4200-960a-b2fa37c6b18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:53.748959 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4e986-d5d1-4200-960a-b2fa37c6b18a None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:53.749242 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4e986-d5d1-4200-960a-b2fa37c6b18a None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:53.749511 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4e986-d5d1-4200-960a-b2fa37c6b18a None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:53.750220 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4e986-d5d1-4200-960a-b2fa37c6b18a None admin] RBAC: Adding query filter params (name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:53.750522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4e986-d5d1-4200-960a-b2fa37c6b18a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:53.751922 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4e986-d5d1-4200-960a-b2fa37c6b18a None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:53.760287 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 259/512] 10.222.0.73 () {56 vars in 1103 bytes} [Mon Aug 29 19:59:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:53.761025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4e986-d5d1-4200-960a-b2fa37c6b18a None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:53.788652 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:53.815373 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:53.849137 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f4b33-bf25-4602-8366-065d512a1365 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:53.873919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4e986-d5d1-4200-960a-b2fa37c6b18a None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:53.885064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50308f1-ac33-41d9-9f4e-4a046d5264d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:53.885667 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50308f1-ac33-41d9-9f4e-4a046d5264d1 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:53.885979 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50308f1-ac33-41d9-9f4e-4a046d5264d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:53.886247 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50308f1-ac33-41d9-9f4e-4a046d5264d1 None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:53.886968 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50308f1-ac33-41d9-9f4e-4a046d5264d1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:53.887307 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50308f1-ac33-41d9-9f4e-4a046d5264d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:53.889341 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50308f1-ac33-41d9-9f4e-4a046d5264d1 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:53.906935 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 254/513] 10.222.0.73 () {56 vars in 1137 bytes} [Mon Aug 29 19:59:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:53.907760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50308f1-ac33-41d9-9f4e-4a046d5264d1 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:53.911820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0089e-e6e7-4938-aad8-57364fdd4697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:53.912434 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0089e-e6e7-4938-aad8-57364fdd4697 None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:53.912709 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0089e-e6e7-4938-aad8-57364fdd4697 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:53.912984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0089e-e6e7-4938-aad8-57364fdd4697 None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:53.913625 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c0089e-e6e7-4938-aad8-57364fdd4697 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:53.915081 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c0089e-e6e7-4938-aad8-57364fdd4697 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:53.936248 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 260/514] 10.222.0.73 () {60 vars in 1132 bytes} [Mon Aug 29 19:59:53 2022] POST /identity/v3/users => generated 247 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:59:53.937213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0089e-e6e7-4938-aad8-57364fdd4697 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:54.036270 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:54.038195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50308f1-ac33-41d9-9f4e-4a046d5264d1 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:54.064752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0089e-e6e7-4938-aad8-57364fdd4697 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:54.070268 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:54.080146 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b805935-47d3-4e82-9e82-fab5902523ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:54.080866 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b805935-47d3-4e82-9e82-fab5902523ba None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:54.081752 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b805935-47d3-4e82-9e82-fab5902523ba None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:54.081752 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b805935-47d3-4e82-9e82-fab5902523ba None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:54.089579 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b805935-47d3-4e82-9e82-fab5902523ba None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:54.091846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b805935-47d3-4e82-9e82-fab5902523ba None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:54.105753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-1b805935-47d3-4e82-9e82-fab5902523ba None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 19:59:54.105753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:54.105753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:54.105753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:54.105753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:54.105753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:54.105753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:54.105753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:54.105753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:54.105753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:54.105753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:54.105753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:54.105753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 19:59:54.105753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 19:59:54.105753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 19:59:54.105753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 19:59:54.107335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:54.107335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:54.107335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 19:59:54.107335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:59:54.107335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 19:59:54.107335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:59:54.107335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:54.107335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:54.107335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:54.107335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:54.107335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:54.107335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:54.107335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:54.107335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:54.107335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:54.107335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:54.107335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:54.107335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:54.108948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:54.108948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:54.108948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 19:59:54.108948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 19:59:54.108948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 19:59:54.108948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 19:59:54.108948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 19:59:54.108948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 19:59:54.108948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 19:59:54.108948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:54.108948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 255/515] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 19:59:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:54.109551 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da9503-deec-4b12-a1a2-c62bd2db79d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:54.109646 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b805935-47d3-4e82-9e82-fab5902523ba None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:54.109733 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da9503-deec-4b12-a1a2-c62bd2db79d3 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:54.109733 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da9503-deec-4b12-a1a2-c62bd2db79d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:54.110100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da9503-deec-4b12-a1a2-c62bd2db79d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:54.117301 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da9503-deec-4b12-a1a2-c62bd2db79d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:54.118824 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da9503-deec-4b12-a1a2-c62bd2db79d3 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:54.133534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-67da9503-deec-4b12-a1a2-c62bd2db79d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 19:59:54.133534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:54.133534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:54.133534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:54.133534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:54.133534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:54.133534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:54.133534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:54.133534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:54.133534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:54.133534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:54.133534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:54.133534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 19:59:54.133534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 19:59:54.133534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 19:59:54.134650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 19:59:54.134650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:54.134650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:54.134650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:54.134650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:54.134650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:54.134650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:54.134650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:54.134650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:54.134650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:54.134650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:54.134650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:54.134650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:54.134650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:54.134650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:54.134650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:54.134650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:54.136455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 19:59:54.136455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 19:59:54.136455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 19:59:54.136455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 19:59:54.136455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 19:59:54.136455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 19:59:54.136455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 19:59:54.136455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:54.136455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 261/516] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 19:59:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:54.136455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da9503-deec-4b12-a1a2-c62bd2db79d3 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:54.227571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b805935-47d3-4e82-9e82-fab5902523ba None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:54.249390 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da9503-deec-4b12-a1a2-c62bd2db79d3 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:54.265310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f604e8-f755-4f1b-8360-b696c3c299e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:54.266236 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f604e8-f755-4f1b-8360-b696c3c299e9 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:54.266686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f604e8-f755-4f1b-8360-b696c3c299e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:54.267189 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f604e8-f755-4f1b-8360-b696c3c299e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:54.279869 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f604e8-f755-4f1b-8360-b696c3c299e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:54.281887 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f604e8-f755-4f1b-8360-b696c3c299e9 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:54.285747 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20836577-3678-46cd-aa0e-c70ccb695bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:54.286373 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20836577-3678-46cd-aa0e-c70ccb695bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:54.286653 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20836577-3678-46cd-aa0e-c70ccb695bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:54.286928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20836577-3678-46cd-aa0e-c70ccb695bf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:54.299273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-e2f604e8-f755-4f1b-8360-b696c3c299e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:54.299273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:54.299273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:54.299273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:54.299273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:54.299273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:54.299273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:54.299273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:54.299273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:54.299273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:54.299273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:54.299273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:54.299273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:59:54.299273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:59:54.299273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:59:54.299273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:59:54.300402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:54.300402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:54.300402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:54.300402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:54.300402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:54.300402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:54.300402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:54.300402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:54.300402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:54.300402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:54.300402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:54.300402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:54.300402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:54.300402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:54.300402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:54.300402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:54.300402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:59:54.301424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:59:54.301424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:54.301424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:54.302035 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:54.303303 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 256/517] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:54.304677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f604e8-f755-4f1b-8360-b696c3c299e9 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:54.352629 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:54.371696 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20836577-3678-46cd-aa0e-c70ccb695bf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:54.373274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20836577-3678-46cd-aa0e-c70ccb695bf7 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:54.424275 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f604e8-f755-4f1b-8360-b696c3c299e9 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:54.463736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87544a-7d45-4a99-96e6-315c8d855c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:54.464562 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87544a-7d45-4a99-96e6-315c8d855c86 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:54.464870 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87544a-7d45-4a99-96e6-315c8d855c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:54.465184 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87544a-7d45-4a99-96e6-315c8d855c86 None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:54.466598 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a87544a-7d45-4a99-96e6-315c8d855c86 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:54.466902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a87544a-7d45-4a99-96e6-315c8d855c86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:54.468429 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a87544a-7d45-4a99-96e6-315c8d855c86 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:54.489018 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 262/518] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:54.489018 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20836577-3678-46cd-aa0e-c70ccb695bf7 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:54.493690 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 257/519] 10.222.0.73 () {56 vars in 1137 bytes} [Mon Aug 29 19:59:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:54.498790 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87544a-7d45-4a99-96e6-315c8d855c86 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:54.538880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:54.605554 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:54.609256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20836577-3678-46cd-aa0e-c70ccb695bf7 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:54.610364 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87544a-7d45-4a99-96e6-315c8d855c86 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:54.647662 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ad84e-82eb-4c52-86d7-ab0dc4e061a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:54.648337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ad84e-82eb-4c52-86d7-ab0dc4e061a8 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:54.648550 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ad84e-82eb-4c52-86d7-ab0dc4e061a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:54.648723 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ad84e-82eb-4c52-86d7-ab0dc4e061a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:54.649420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ad84e-82eb-4c52-86d7-ab0dc4e061a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:54.649736 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ad84e-82eb-4c52-86d7-ab0dc4e061a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:54.651223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ad84e-82eb-4c52-86d7-ab0dc4e061a8 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:54.655824 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007a39c-99dd-40d7-be8f-2816222c61ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:54.656443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007a39c-99dd-40d7-be8f-2816222c61ec None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:54.656721 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007a39c-99dd-40d7-be8f-2816222c61ec None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:54.656989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007a39c-99dd-40d7-be8f-2816222c61ec None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:54.657626 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007a39c-99dd-40d7-be8f-2816222c61ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:54.658016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007a39c-99dd-40d7-be8f-2816222c61ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:54.659459 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007a39c-99dd-40d7-be8f-2816222c61ec None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:54.671943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 263/520] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 19:59:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:54.676832 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ad84e-82eb-4c52-86d7-ab0dc4e061a8 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:54.676989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 258/521] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 19:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:54.676989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007a39c-99dd-40d7-be8f-2816222c61ec None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:54.785599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007a39c-99dd-40d7-be8f-2816222c61ec None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:54.789268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:54.792323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ad84e-82eb-4c52-86d7-ab0dc4e061a8 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:54.822883 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860de571-66c8-48cf-a9d0-d4afd0501850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:54.823499 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860de571-66c8-48cf-a9d0-d4afd0501850 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:54.823775 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860de571-66c8-48cf-a9d0-d4afd0501850 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:54.824055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860de571-66c8-48cf-a9d0-d4afd0501850 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:54.833136 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038fd416-0b7e-4ce2-90ba-36a6346e2f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:54.833801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fd416-0b7e-4ce2-90ba-36a6346e2f54 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:54.834079 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fd416-0b7e-4ce2-90ba-36a6346e2f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:54.834347 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fd416-0b7e-4ce2-90ba-36a6346e2f54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:54.846326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038fd416-0b7e-4ce2-90ba-36a6346e2f54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:54.849010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038fd416-0b7e-4ce2-90ba-36a6346e2f54 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:54.856993 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:54.868830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-038fd416-0b7e-4ce2-90ba-36a6346e2f54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:54.868830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:54.868830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:54.868830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:54.868830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:54.868830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:54.868830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:54.868830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:54.868830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:54.868830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:54.868830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:54.868830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:54.868830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:59:54.868830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:59:54.868830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:59:54.868830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:59:54.869860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:54.869860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:54.869860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:54.869860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:54.869860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:54.869860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:54.869860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:54.869860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:54.869860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:54.869860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:54.869860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:54.869860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:54.869860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:54.869860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:54.869860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:54.869860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:54.869860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:59:54.876714 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:59:54.876714 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:54.876714 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:54.876714 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 264/522] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:54.876714 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038fd416-0b7e-4ce2-90ba-36a6346e2f54 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:54.908642 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860de571-66c8-48cf-a9d0-d4afd0501850 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:54.910055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860de571-66c8-48cf-a9d0-d4afd0501850 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:55.002001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038fd416-0b7e-4ce2-90ba-36a6346e2f54 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:55.032216 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 259/523] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:55.037444 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860de571-66c8-48cf-a9d0-d4afd0501850 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:55.041442 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85da882-3a64-4748-a17e-496676d93273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:55.042117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85da882-3a64-4748-a17e-496676d93273 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:55.042398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85da882-3a64-4748-a17e-496676d93273 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:55.042658 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85da882-3a64-4748-a17e-496676d93273 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:55.051134 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85da882-3a64-4748-a17e-496676d93273 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:55.052591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85da882-3a64-4748-a17e-496676d93273 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:55.066280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-f85da882-3a64-4748-a17e-496676d93273 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 19:59:55.066280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:55.066280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:55.066280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:55.066280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:55.066280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:55.066280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:55.066280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:55.066280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:55.066280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:55.066280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:55.066280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:55.066280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 19:59:55.066280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 19:59:55.066280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 19:59:55.066280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 19:59:55.068226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:55.068226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:55.068226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 19:59:55.068226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:59:55.068226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 19:59:55.068226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:59:55.068226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:55.068226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:55.068226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:55.068226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:55.068226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:55.068226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:55.068226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:55.068226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:55.068226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:55.068226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:55.068226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:55.068226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:55.069283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:55.069283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:55.069283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 19:59:55.069283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 19:59:55.069283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 19:59:55.069283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 19:59:55.069283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 19:59:55.069283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 19:59:55.069283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 19:59:55.069283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:55.069283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 265/524] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 19:59:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:55.069283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85da882-3a64-4748-a17e-496676d93273 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:55.090426 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:55.108660 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:55.162424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860de571-66c8-48cf-a9d0-d4afd0501850 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:55.202892 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85da882-3a64-4748-a17e-496676d93273 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:55.205452 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae903bef-fd3a-491e-86f4-4478baad17a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:55.206152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae903bef-fd3a-491e-86f4-4478baad17a7 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:55.206496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae903bef-fd3a-491e-86f4-4478baad17a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:55.206927 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae903bef-fd3a-491e-86f4-4478baad17a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:55.207710 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae903bef-fd3a-491e-86f4-4478baad17a7 None admin] RBAC: Adding query filter params (role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=95cf77a92a174ff8b0503baf5c7e63be) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:55.208015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae903bef-fd3a-491e-86f4-4478baad17a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:55.209451 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae903bef-fd3a-491e-86f4-4478baad17a7 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:55.220620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 260/525] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 19:59:55 2022] GET /identity/v3/role_assignments?user.id=95cf77a92a174ff8b0503baf5c7e63be&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:55.234780 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae903bef-fd3a-491e-86f4-4478baad17a7 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:55.252901 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd234b-ce95-462a-b9c8-0f8d6a7e88c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:55.254712 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd234b-ce95-462a-b9c8-0f8d6a7e88c0 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:55.254712 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd234b-ce95-462a-b9c8-0f8d6a7e88c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:55.255119 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd234b-ce95-462a-b9c8-0f8d6a7e88c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:55.256197 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd234b-ce95-462a-b9c8-0f8d6a7e88c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:55.256707 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd234b-ce95-462a-b9c8-0f8d6a7e88c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:55.259307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd234b-ce95-462a-b9c8-0f8d6a7e88c0 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:55.287975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 266/526] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 19:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:55.288988 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd234b-ce95-462a-b9c8-0f8d6a7e88c0 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:55.381045 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:55.409314 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae903bef-fd3a-491e-86f4-4478baad17a7 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:55.427381 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd234b-ce95-462a-b9c8-0f8d6a7e88c0 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:55.434481 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:55.449134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10437f0b-8dfe-49b4-b9f8-9ab1e2038ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:55.450602 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10437f0b-8dfe-49b4-b9f8-9ab1e2038ebf None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:55.450784 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10437f0b-8dfe-49b4-b9f8-9ab1e2038ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:55.451263 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10437f0b-8dfe-49b4-b9f8-9ab1e2038ebf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:55.464317 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10437f0b-8dfe-49b4-b9f8-9ab1e2038ebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:55.466841 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10437f0b-8dfe-49b4-b9f8-9ab1e2038ebf None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:55.478222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb8678-c8f1-4387-b3bc-c509f5634ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:55.479146 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb8678-c8f1-4387-b3bc-c509f5634ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:55.479607 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb8678-c8f1-4387-b3bc-c509f5634ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:55.480060 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb8678-c8f1-4387-b3bc-c509f5634ea2 None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:55.480971 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb8678-c8f1-4387-b3bc-c509f5634ea2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:55.481457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb8678-c8f1-4387-b3bc-c509f5634ea2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:55.483117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb8678-c8f1-4387-b3bc-c509f5634ea2 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:55.483534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-10437f0b-8dfe-49b4-b9f8-9ab1e2038ebf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:55.483534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:55.483534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:55.483534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:55.483534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:55.483534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:55.483534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:55.483534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:55.483534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:55.483534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:55.483534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:55.483534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:55.483534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:59:55.483534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:59:55.483534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:59:55.483534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:59:55.484416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:55.484416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:55.484416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:55.484416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:55.484416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:55.484416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:55.484416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:55.484416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:55.484416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:55.484416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:55.484416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:55.484416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:55.484416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:55.484416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:55.484416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:55.484416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:55.484416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:59:55.485252 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:59:55.485252 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:55.485252 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:55.485252 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 261/527] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:55.486349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10437f0b-8dfe-49b4-b9f8-9ab1e2038ebf None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:55.541315 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 267/528] 10.222.0.73 () {56 vars in 1139 bytes} [Mon Aug 29 19:59:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:55.542934 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb8678-c8f1-4387-b3bc-c509f5634ea2 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:55.609290 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10437f0b-8dfe-49b4-b9f8-9ab1e2038ebf None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:55.654356 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fdd35a-6b37-4922-89e4-958724735bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:55.655148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdd35a-6b37-4922-89e4-958724735bac None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:55.655512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdd35a-6b37-4922-89e4-958724735bac None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:55.655880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdd35a-6b37-4922-89e4-958724735bac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:55.663717 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fdd35a-6b37-4922-89e4-958724735bac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:55.665317 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fdd35a-6b37-4922-89e4-958724735bac None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:55.667950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb8678-c8f1-4387-b3bc-c509f5634ea2 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:55.680093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-58fdd35a-6b37-4922-89e4-958724735bac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 19:59:55.680093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:55.680093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:55.680093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:55.680093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:55.680093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:55.680093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:55.680093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:55.680093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:55.680093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:55.680093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:55.680093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:55.680093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 19:59:55.680093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 19:59:55.680093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 19:59:55.681008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 19:59:55.681008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:55.681008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:55.681008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:55.681008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:55.681008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:55.681008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:55.681008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:55.681008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:55.681008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:55.681008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:55.681008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:55.681008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:55.681008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:55.681008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:55.681008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:55.681008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:55.681882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 19:59:55.681882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 19:59:55.681882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 19:59:55.681882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 19:59:55.681882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 19:59:55.681882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 19:59:55.681882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 19:59:55.681882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:55.681882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 262/529] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 19:59:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:55.683078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fdd35a-6b37-4922-89e4-958724735bac None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:55.706285 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:55.714285 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb0f64-f0ac-4cdd-9a8c-6d78b7327c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:55.715231 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb0f64-f0ac-4cdd-9a8c-6d78b7327c22 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:55.715887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb0f64-f0ac-4cdd-9a8c-6d78b7327c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:55.716575 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb0f64-f0ac-4cdd-9a8c-6d78b7327c22 None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:55.717455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb0f64-f0ac-4cdd-9a8c-6d78b7327c22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:55.717973 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb0f64-f0ac-4cdd-9a8c-6d78b7327c22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:55.719755 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb0f64-f0ac-4cdd-9a8c-6d78b7327c22 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:55.722243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:55.735449 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 268/530] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 19:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:55.736717 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb0f64-f0ac-4cdd-9a8c-6d78b7327c22 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:55.793275 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fdd35a-6b37-4922-89e4-958724735bac None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:55.847630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a4e56-2da1-4e7d-a533-cdae9b177aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:55.849273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a4e56-2da1-4e7d-a533-cdae9b177aee None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:55.849670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a4e56-2da1-4e7d-a533-cdae9b177aee None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:55.850045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a4e56-2da1-4e7d-a533-cdae9b177aee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:55.864992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb0f64-f0ac-4cdd-9a8c-6d78b7327c22 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:55.906508 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f7e36-d628-4c69-bfdb-6fe75c294c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:55.907305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f7e36-d628-4c69-bfdb-6fe75c294c25 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:55.907678 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f7e36-d628-4c69-bfdb-6fe75c294c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:55.908048 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f7e36-d628-4c69-bfdb-6fe75c294c25 None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:55.908831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f7e36-d628-4c69-bfdb-6fe75c294c25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:55.909233 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f7e36-d628-4c69-bfdb-6fe75c294c25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:55.910796 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f7e36-d628-4c69-bfdb-6fe75c294c25 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:55.924739 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 269/531] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 19:59:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:55.925798 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f7e36-d628-4c69-bfdb-6fe75c294c25 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:55.947398 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a4e56-2da1-4e7d-a533-cdae9b177aee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:55.948905 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a4e56-2da1-4e7d-a533-cdae9b177aee None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:55.978198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:55.991441 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:56.034230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f7e36-d628-4c69-bfdb-6fe75c294c25 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:56.069918 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 263/532] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:56.070769 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a4e56-2da1-4e7d-a533-cdae9b177aee None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:56.080922 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f554459-b4d0-4580-a7fa-7181419d5752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:56.081939 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f554459-b4d0-4580-a7fa-7181419d5752 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:56.082219 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f554459-b4d0-4580-a7fa-7181419d5752 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:56.082904 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f554459-b4d0-4580-a7fa-7181419d5752 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:56.090713 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f554459-b4d0-4580-a7fa-7181419d5752 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:56.092567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f554459-b4d0-4580-a7fa-7181419d5752 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:56.105223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-1f554459-b4d0-4580-a7fa-7181419d5752 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 19:59:56.105223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:56.105223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:56.105223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:56.105223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:56.105223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:56.105223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:56.105223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:56.105223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:56.105223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:56.105223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:56.105223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:56.105223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 19:59:56.105223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 19:59:56.105223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 19:59:56.106889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 19:59:56.106889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:56.106889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:56.106889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:56.106889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:56.106889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:56.106889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:56.106889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:56.106889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:56.106889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:56.106889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:56.106889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:56.106889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:56.106889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:56.106889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:56.106889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:56.106889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:56.112944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 19:59:56.112944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 19:59:56.112944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 19:59:56.112944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 19:59:56.112944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 19:59:56.112944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 19:59:56.112944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 19:59:56.112944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:56.112944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 270/533] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 19:59:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:56.112944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37798b6-808a-4694-8b13-61890d524378 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:56.112944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37798b6-808a-4694-8b13-61890d524378 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:56.113522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37798b6-808a-4694-8b13-61890d524378 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:56.114514 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 271/534] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:56.115307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37798b6-808a-4694-8b13-61890d524378 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:56.186632 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a4e56-2da1-4e7d-a533-cdae9b177aee None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:56.223054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce2bbc-cbd4-46de-a2ca-9cd29be0dca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:56.223579 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce2bbc-cbd4-46de-a2ca-9cd29be0dca4 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:56.223859 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce2bbc-cbd4-46de-a2ca-9cd29be0dca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:56.224132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce2bbc-cbd4-46de-a2ca-9cd29be0dca4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:56.224857 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce2bbc-cbd4-46de-a2ca-9cd29be0dca4 None admin] RBAC: Adding query filter params (role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=103b64b4b172467692d02851b2955b81) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:56.224955 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37798b6-808a-4694-8b13-61890d524378 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:56.225283 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce2bbc-cbd4-46de-a2ca-9cd29be0dca4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:56.226778 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce2bbc-cbd4-46de-a2ca-9cd29be0dca4 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:56.228788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:56.237598 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:56.246072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 264/535] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 19:59:56 2022] GET /identity/v3/role_assignments?user.id=103b64b4b172467692d02851b2955b81&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:56.246960 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce2bbc-cbd4-46de-a2ca-9cd29be0dca4 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:56.266257 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0aa546-3aff-4a9e-846f-1aba88d68c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:56.266915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0aa546-3aff-4a9e-846f-1aba88d68c23 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:56.267192 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0aa546-3aff-4a9e-846f-1aba88d68c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:56.267459 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0aa546-3aff-4a9e-846f-1aba88d68c23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:56.280088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0aa546-3aff-4a9e-846f-1aba88d68c23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:56.281586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0aa546-3aff-4a9e-846f-1aba88d68c23 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:56.299305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-4f0aa546-3aff-4a9e-846f-1aba88d68c23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:56.299305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:56.299305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:56.299305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:56.299305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:56.299305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:56.299305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:56.299305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:56.299305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:56.299305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:56.299305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:56.299305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:56.299305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:59:56.299305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:59:56.299305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:59:56.299305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:59:56.300292 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:56.300292 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:56.300292 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:56.300292 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:56.300292 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:56.300292 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:56.300292 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:56.300292 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:56.300292 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:56.300292 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:56.300292 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:56.300292 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:56.300292 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:56.300292 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:56.300292 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:56.300292 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:56.300292 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:59:56.301129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:59:56.301129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:56.301129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:56.301129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 272/536] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:56.302215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e089d7-196c-41b8-afa7-8184be25a9eb None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:56.302762 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e089d7-196c-41b8-afa7-8184be25a9eb None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:56.302862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e089d7-196c-41b8-afa7-8184be25a9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:56.323806 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-d4e089d7-196c-41b8-afa7-8184be25a9eb None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:56.326250 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-d4e089d7-196c-41b8-afa7-8184be25a9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:56.414169 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce2bbc-cbd4-46de-a2ca-9cd29be0dca4 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:56.425730 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 273/537] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:56.425730 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e089d7-196c-41b8-afa7-8184be25a9eb None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:56.483365 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbdc2b-13e4-43bd-b791-21f577250295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:56.484318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbdc2b-13e4-43bd-b791-21f577250295 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:56.484791 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbdc2b-13e4-43bd-b791-21f577250295 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:56.485360 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbdc2b-13e4-43bd-b791-21f577250295 None admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:56.486397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bbdc2b-13e4-43bd-b791-21f577250295 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:56.486912 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bbdc2b-13e4-43bd-b791-21f577250295 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:56.488810 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bbdc2b-13e4-43bd-b791-21f577250295 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:56.527760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 265/538] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 19:59:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:56.535148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbdc2b-13e4-43bd-b791-21f577250295 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:56.535797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:56.553205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e089d7-196c-41b8-afa7-8184be25a9eb None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:56.636987 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:56.654580 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1be6f-fd66-4dc7-9797-c24fd174c496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:56.655214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1be6f-fd66-4dc7-9797-c24fd174c496 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:56.655492 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1be6f-fd66-4dc7-9797-c24fd174c496 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:56.655762 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1be6f-fd66-4dc7-9797-c24fd174c496 None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:56.656414 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1be6f-fd66-4dc7-9797-c24fd174c496 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:56.656715 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1be6f-fd66-4dc7-9797-c24fd174c496 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:56.658161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1be6f-fd66-4dc7-9797-c24fd174c496 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:56.684406 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbdc2b-13e4-43bd-b791-21f577250295 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:56.688219 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 274/539] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 19:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:56.693476 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1be6f-fd66-4dc7-9797-c24fd174c496 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:56.738671 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855e0f6-36d8-4b18-b062-8de892cd1f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:56.739202 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855e0f6-36d8-4b18-b062-8de892cd1f2e None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:56.739480 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855e0f6-36d8-4b18-b062-8de892cd1f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:56.739748 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855e0f6-36d8-4b18-b062-8de892cd1f2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:56.740474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855e0f6-36d8-4b18-b062-8de892cd1f2e None admin] RBAC: Adding query filter params (role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=4a0b2ca8d14a47239a26912e58348f5a) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:56.740776 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855e0f6-36d8-4b18-b062-8de892cd1f2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:56.742262 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855e0f6-36d8-4b18-b062-8de892cd1f2e None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:56.757476 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 266/540] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 19:59:56 2022] GET /identity/v3/role_assignments?user.id=4a0b2ca8d14a47239a26912e58348f5a&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:56.769869 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855e0f6-36d8-4b18-b062-8de892cd1f2e None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:56.914048 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1be6f-fd66-4dc7-9797-c24fd174c496 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:56.971137 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855e0f6-36d8-4b18-b062-8de892cd1f2e None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:57.000087 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad6d43-2c38-4ec7-8f51-a76463a19e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:57.001075 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:57.002734 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad6d43-2c38-4ec7-8f51-a76463a19e87 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:57.003530 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad6d43-2c38-4ec7-8f51-a76463a19e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:57.004709 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad6d43-2c38-4ec7-8f51-a76463a19e87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:57.013053 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad6d43-2c38-4ec7-8f51-a76463a19e87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:57.015215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad6d43-2c38-4ec7-8f51-a76463a19e87 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:57.035076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-e8ad6d43-2c38-4ec7-8f51-a76463a19e87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 19:59:57.035076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:57.035076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:57.035076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:57.035076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:57.035076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:57.035076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:57.035076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:57.035076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:57.035076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:57.035076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:57.035076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:57.035076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 19:59:57.035076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 19:59:57.035076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 19:59:57.035967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 19:59:57.035967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:57.035967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:57.035967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:57.035967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:57.035967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:57.035967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:57.035967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:57.035967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:57.035967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:57.035967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:57.035967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:57.035967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:57.035967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:57.035967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:57.035967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:57.035967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:57.036794 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 19:59:57.036794 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 19:59:57.036794 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 19:59:57.036794 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 19:59:57.036794 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 19:59:57.036794 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 19:59:57.036794 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 19:59:57.036794 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:57.038317 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e3a96-a552-4679-a214-71a3180ded35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:57.039260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e3a96-a552-4679-a214-71a3180ded35 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:57.039761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e3a96-a552-4679-a214-71a3180ded35 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:57.040217 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e3a96-a552-4679-a214-71a3180ded35 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:57.042147 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 275/541] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 19:59:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:57.048160 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad6d43-2c38-4ec7-8f51-a76463a19e87 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:57.066063 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e3a96-a552-4679-a214-71a3180ded35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:57.074873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-133e3a96-a552-4679-a214-71a3180ded35 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 19:59:57.074873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:57.074873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:57.074873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:57.074873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:57.074873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:57.074873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:57.074873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:57.074873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:57.074873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:57.074873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:57.074873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:57.074873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:57.074873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:57.074873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:57.074873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:57.076754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:57.076754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:57.076754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:57.076754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:57.076754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:57.076754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:57.076754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:57.076754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:57.076754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:57.076754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:57.076754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:57.076754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:57.076754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:57.076754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:57.076754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:57.076754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:57.077525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:57.077525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:57.077525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:57.077525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:57.077525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:57.077525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:57.077525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:57.077525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:57.077525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:57.077525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:57.077525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 19:59:57.077525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:57.080618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 267/542] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 19:59:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 19:59:57.093728 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:57.093922 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e3a96-a552-4679-a214-71a3180ded35 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:57.274569 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad6d43-2c38-4ec7-8f51-a76463a19e87 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:57.307110 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e3a96-a552-4679-a214-71a3180ded35 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:57.321977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984e46e-01f6-48c0-a7cc-6e134abedada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:57.323099 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984e46e-01f6-48c0-a7cc-6e134abedada None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:57.323387 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984e46e-01f6-48c0-a7cc-6e134abedada None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:57.323819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984e46e-01f6-48c0-a7cc-6e134abedada None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:57.324668 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7984e46e-01f6-48c0-a7cc-6e134abedada None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:57.325015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7984e46e-01f6-48c0-a7cc-6e134abedada None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:57.328303 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7984e46e-01f6-48c0-a7cc-6e134abedada None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:57.349518 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0bc2c-e17c-4870-82ef-ec133e1c34e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:57.350420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0bc2c-e17c-4870-82ef-ec133e1c34e2 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:57.350816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0bc2c-e17c-4870-82ef-ec133e1c34e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:57.351196 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0bc2c-e17c-4870-82ef-ec133e1c34e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:57.351353 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 276/543] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 19:59:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:57.352280 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0bc2c-e17c-4870-82ef-ec133e1c34e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:57.352628 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb778668-f996-47ba-b396-afaccb0f5c54 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:57.352896 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0bc2c-e17c-4870-82ef-ec133e1c34e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:57.353014 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb778668-f996-47ba-b396-afaccb0f5c54 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:57.353432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb778668-f996-47ba-b396-afaccb0f5c54 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:57.354955 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0bc2c-e17c-4870-82ef-ec133e1c34e2 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:57.355194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 277/544] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:57.359498 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb778668-f996-47ba-b396-afaccb0f5c54 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:57.375424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 268/545] 10.222.0.73 () {56 vars in 1103 bytes} [Mon Aug 29 19:59:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:57.377411 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36ff99-b313-4f0a-9c4a-11a7c965c9d0 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:57.377814 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36ff99-b313-4f0a-9c4a-11a7c965c9d0 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:57.378286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36ff99-b313-4f0a-9c4a-11a7c965c9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:57.400087 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-ae36ff99-b313-4f0a-9c4a-11a7c965c9d0 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:57.401651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-ae36ff99-b313-4f0a-9c4a-11a7c965c9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:57.445675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:57.456046 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:57.486699 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 269/546] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:57.488517 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36ff99-b313-4f0a-9c4a-11a7c965c9d0 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:57.501033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb778668-f996-47ba-b396-afaccb0f5c54 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:57.546937 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb830e26-acd9-4a7a-842d-20aeb95ff6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:57.547615 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb830e26-acd9-4a7a-842d-20aeb95ff6f6 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:57.547999 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb830e26-acd9-4a7a-842d-20aeb95ff6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:57.548734 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb830e26-acd9-4a7a-842d-20aeb95ff6f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:57.549902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb830e26-acd9-4a7a-842d-20aeb95ff6f6 None admin] RBAC: Adding query filter params (role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=64ba58d774674ed7b233ad6416db068a) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:57.550341 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb830e26-acd9-4a7a-842d-20aeb95ff6f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:57.552496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb830e26-acd9-4a7a-842d-20aeb95ff6f6 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:57.563417 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 278/547] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 19:59:57 2022] GET /identity/v3/role_assignments?user.id=64ba58d774674ed7b233ad6416db068a&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:57.615614 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36ff99-b313-4f0a-9c4a-11a7c965c9d0 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:57.656410 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab242beb-c0b6-458d-85a6-b7e7e655b795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:57.657266 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab242beb-c0b6-458d-85a6-b7e7e655b795 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:57.657676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab242beb-c0b6-458d-85a6-b7e7e655b795 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:57.658082 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab242beb-c0b6-458d-85a6-b7e7e655b795 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:57.669506 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab242beb-c0b6-458d-85a6-b7e7e655b795 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:57.671290 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab242beb-c0b6-458d-85a6-b7e7e655b795 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:57.688049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-ab242beb-c0b6-458d-85a6-b7e7e655b795 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:57.688049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:57.688049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:57.688049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:57.688049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:57.688049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:57.688049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:57.688049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:57.688049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:57.688049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:57.688049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:57.688049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:57.688049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:59:57.688049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:59:57.688049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:59:57.688049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:59:57.689170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:57.689170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:57.689170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:57.689170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:57.689170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:57.689170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:57.689170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:57.689170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:57.689170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:57.689170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:57.689170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:57.689170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:57.689170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:57.689170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:57.689170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:57.689170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:57.689170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:59:57.690237 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:59:57.690237 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:57.690237 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:57.691844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 270/548] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:57.697286 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb830e26-acd9-4a7a-842d-20aeb95ff6f6 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:57.823094 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb830e26-acd9-4a7a-842d-20aeb95ff6f6 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:57.827428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab242beb-c0b6-458d-85a6-b7e7e655b795 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:57.859540 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:57.866107 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ae73a-1f70-418d-a5b7-b1efac080f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:57.866788 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ae73a-1f70-418d-a5b7-b1efac080f8c None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:57.867224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ae73a-1f70-418d-a5b7-b1efac080f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:57.867511 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ae73a-1f70-418d-a5b7-b1efac080f8c None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:57.868163 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ae73a-1f70-418d-a5b7-b1efac080f8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:57.868469 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ae73a-1f70-418d-a5b7-b1efac080f8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:57.870812 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ae73a-1f70-418d-a5b7-b1efac080f8c None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:57.881833 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:57.901232 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 279/549] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 19:59:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:57.922815 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ae73a-1f70-418d-a5b7-b1efac080f8c None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:57.973139 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab242beb-c0b6-458d-85a6-b7e7e655b795 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:58.011320 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bad72e-d185-43fd-bb08-bf2589fbb2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:58.012343 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bad72e-d185-43fd-bb08-bf2589fbb2a5 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:58.013406 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bad72e-d185-43fd-bb08-bf2589fbb2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:58.014057 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bad72e-d185-43fd-bb08-bf2589fbb2a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:58.026212 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bad72e-d185-43fd-bb08-bf2589fbb2a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:58.032709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-75bad72e-d185-43fd-bb08-bf2589fbb2a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 19:59:58.032709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:58.032709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:58.032709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:58.032709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:58.032709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:58.032709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:58.032709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:58.032709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:58.032709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:58.032709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:58.032709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:58.032709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:58.032709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:58.032709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:58.032709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:58.034195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:58.034195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:58.034195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:58.034195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:58.034195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:58.034195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:58.034195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:58.034195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:58.034195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:58.034195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:58.034195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:58.034195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:58.034195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:58.034195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:58.034195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:58.034195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:58.035148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:58.035148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:58.035148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:58.035148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:58.035148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:58.035148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:58.035148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:58.035148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:58.035148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:58.035148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:58.035148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 19:59:58.035148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:58.041888 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 271/550] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 19:59:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:58.043908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bad72e-d185-43fd-bb08-bf2589fbb2a5 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:58.131878 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ae73a-1f70-418d-a5b7-b1efac080f8c None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:58.142107 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:58.162333 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bad72e-d185-43fd-bb08-bf2589fbb2a5 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:58.205240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b785e-4949-4d8e-a73a-38f350826324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:58.206288 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b785e-4949-4d8e-a73a-38f350826324 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:58.206901 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b785e-4949-4d8e-a73a-38f350826324 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:58.207455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b785e-4949-4d8e-a73a-38f350826324 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:58.224158 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b785e-4949-4d8e-a73a-38f350826324 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:58.226765 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b785e-4949-4d8e-a73a-38f350826324 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:58.232658 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce026f48-1e85-4003-b731-cc441f776839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:58.234526 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce026f48-1e85-4003-b731-cc441f776839 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:58.236609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce026f48-1e85-4003-b731-cc441f776839 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:58.239457 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce026f48-1e85-4003-b731-cc441f776839 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:58.240357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce026f48-1e85-4003-b731-cc441f776839 None admin] RBAC: Adding query filter params (name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:58.240867 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce026f48-1e85-4003-b731-cc441f776839 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:58.242441 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce026f48-1e85-4003-b731-cc441f776839 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:58.249271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-5e2b785e-4949-4d8e-a73a-38f350826324 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 19:59:58.249271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:58.249271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:58.249271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:58.249271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:58.249271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:58.249271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:58.249271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:58.249271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:58.249271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:58.249271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:58.249271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:58.249271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 19:59:58.249271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 19:59:58.249271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 19:59:58.249271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 19:59:58.250191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:58.250191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:58.250191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 19:59:58.250191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:59:58.250191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 19:59:58.250191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:59:58.250191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:58.250191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:58.250191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:58.250191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:58.250191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:58.250191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:58.250191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:58.250191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:58.250191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:58.250191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:58.250191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:58.250191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:58.251294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:58.251294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:58.251294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 19:59:58.251294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 19:59:58.251294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 19:59:58.251294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 19:59:58.251294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 19:59:58.251294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 19:59:58.251294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 19:59:58.251294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:58.267778 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 280/552] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 19:59:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 19:59:58.268218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 272/551] 10.222.0.73 () {56 vars in 1103 bytes} [Mon Aug 29 19:59:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:59:58.274769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b785e-4949-4d8e-a73a-38f350826324 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:58.274922 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce026f48-1e85-4003-b731-cc441f776839 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:58.384809 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce026f48-1e85-4003-b731-cc441f776839 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:58.406485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:58.455258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b785e-4949-4d8e-a73a-38f350826324 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:58.478438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa0d39-1ed8-4d4e-8eeb-2340c37b114b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:58.479460 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa0d39-1ed8-4d4e-8eeb-2340c37b114b None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:58.479933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa0d39-1ed8-4d4e-8eeb-2340c37b114b None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:58.480393 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa0d39-1ed8-4d4e-8eeb-2340c37b114b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:58.492546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d07ccb-240d-46f4-881e-bfa66db36f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:58.493159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d07ccb-240d-46f4-881e-bfa66db36f7b None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:58.493443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d07ccb-240d-46f4-881e-bfa66db36f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:58.493738 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d07ccb-240d-46f4-881e-bfa66db36f7b None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:58.494486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d07ccb-240d-46f4-881e-bfa66db36f7b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:58.494794 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d07ccb-240d-46f4-881e-bfa66db36f7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:58.496227 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d07ccb-240d-46f4-881e-bfa66db36f7b None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:58.516573 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aa0d39-1ed8-4d4e-8eeb-2340c37b114b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:58.517281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 281/553] 10.222.0.73 () {56 vars in 1135 bytes} [Mon Aug 29 19:59:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:58.518557 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aa0d39-1ed8-4d4e-8eeb-2340c37b114b None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:58.524946 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d07ccb-240d-46f4-881e-bfa66db36f7b None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:58.538341 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:58.551657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-94aa0d39-1ed8-4d4e-8eeb-2340c37b114b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:58.551657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:58.551657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:58.551657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:58.551657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:58.551657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:58.551657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:58.551657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:58.551657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:58.551657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:58.551657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:58.551657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:58.551657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:59:58.551657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:59:58.551657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:59:58.551657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:59:58.552864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:58.552864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:58.552864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:58.552864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:58.552864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:58.552864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:58.552864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:58.552864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:58.552864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:58.552864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:58.552864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:58.552864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:58.552864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:58.552864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:58.552864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:58.552864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:58.552864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:59:58.553913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:59:58.553913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:58.553913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:58.553913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 273/554] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:58.558587 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa0d39-1ed8-4d4e-8eeb-2340c37b114b None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:58.635022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d07ccb-240d-46f4-881e-bfa66db36f7b None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:58.651588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:58.672713 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce978470-d237-4446-ae09-1387c85c8e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:58.673405 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce978470-d237-4446-ae09-1387c85c8e2b None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:58.673693 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce978470-d237-4446-ae09-1387c85c8e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:58.674016 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce978470-d237-4446-ae09-1387c85c8e2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:58.753936 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa0d39-1ed8-4d4e-8eeb-2340c37b114b None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:58.764321 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce978470-d237-4446-ae09-1387c85c8e2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:58.765839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce978470-d237-4446-ae09-1387c85c8e2b None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:58.801736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ef8f2-96d5-4e5f-946d-050b6818ce3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:58.802619 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ef8f2-96d5-4e5f-946d-050b6818ce3a None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:58.802906 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ef8f2-96d5-4e5f-946d-050b6818ce3a None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:58.803181 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ef8f2-96d5-4e5f-946d-050b6818ce3a None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:58.803823 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ef8f2-96d5-4e5f-946d-050b6818ce3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:58.804128 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ef8f2-96d5-4e5f-946d-050b6818ce3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:58.805513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ef8f2-96d5-4e5f-946d-050b6818ce3a None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:58.819842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 274/555] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 19:59:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:58.821132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ca0a0-cf8a-441c-a04c-cbdd8d328db6 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:58.821423 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ca0a0-cf8a-441c-a04c-cbdd8d328db6 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:58.821725 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ca0a0-cf8a-441c-a04c-cbdd8d328db6 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:58.823197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 275/556] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:58.835143 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc353958-fa9e-475f-83bb-b5814d294f9e None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:58.835504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc353958-fa9e-475f-83bb-b5814d294f9e None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:58.835698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc353958-fa9e-475f-83bb-b5814d294f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:58.860087 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-fc353958-fa9e-475f-83bb-b5814d294f9e None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:58.861461 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-fc353958-fa9e-475f-83bb-b5814d294f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:58.890275 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 282/557] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 19:59:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 19:59:58.891532 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce978470-d237-4446-ae09-1387c85c8e2b None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:58.909153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:58.953827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 276/558] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:58.953827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc353958-fa9e-475f-83bb-b5814d294f9e None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:58.958027 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:59.058980 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce978470-d237-4446-ae09-1387c85c8e2b None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:59.083819 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc353958-fa9e-475f-83bb-b5814d294f9e None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:59.108876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ffc23-2284-48f7-ab2c-78a98cd00870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:59.109944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ffc23-2284-48f7-ab2c-78a98cd00870 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:59.110390 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ffc23-2284-48f7-ab2c-78a98cd00870 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:59.110827 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ffc23-2284-48f7-ab2c-78a98cd00870 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:59.119028 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362ffc23-2284-48f7-ab2c-78a98cd00870 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:59.120914 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362ffc23-2284-48f7-ab2c-78a98cd00870 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:59.126544 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272a81b-2e94-496b-bd75-5f0c4df61af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:59.127183 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272a81b-2e94-496b-bd75-5f0c4df61af6 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:59.127462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272a81b-2e94-496b-bd75-5f0c4df61af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:59.127733 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272a81b-2e94-496b-bd75-5f0c4df61af6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:59.135425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-362ffc23-2284-48f7-ab2c-78a98cd00870 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 19:59:59.135425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:59.135425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:59.135425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:59.135425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:59.135425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:59.135425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:59.135425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:59.135425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:59.135425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:59.135425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:59.135425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:59.135425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 19:59:59.135425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 19:59:59.135425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 19:59:59.135425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 19:59:59.136565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:59.136565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:59.136565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 19:59:59.136565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:59:59.136565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 19:59:59.136565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 19:59:59.136565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:59.136565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:59.136565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:59.136565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:59.136565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:59.136565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:59.136565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:59.136565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:59.136565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:59.136565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:59.136565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:59.136565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:59.137949 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:59.137949 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:59.137949 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 19:59:59.137949 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 19:59:59.137949 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 19:59:59.137949 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 19:59:59.137949 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 19:59:59.137949 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 19:59:59.137949 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 19:59:59.137949 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:59.138822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272a81b-2e94-496b-bd75-5f0c4df61af6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:59.139997 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272a81b-2e94-496b-bd75-5f0c4df61af6 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:59.140800 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 283/559] 10.222.0.73 () {56 vars in 1093 bytes} [Mon Aug 29 19:59:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:59.142377 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987eb8c-83da-45b6-a807-b24d064791d6 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:59.143096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987eb8c-83da-45b6-a807-b24d064791d6 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:59.143520 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987eb8c-83da-45b6-a807-b24d064791d6 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:59.149109 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 284/560] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 19:59:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:59:59.150287 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987eb8c-83da-45b6-a807-b24d064791d6 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:59.157465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-6272a81b-2e94-496b-bd75-5f0c4df61af6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:59.157465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:59.157465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:59.157465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:59.157465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:59.157465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:59.157465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:59.157465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:59.157465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:59.157465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:59.157465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:59.157465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:59.157465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:59:59.157465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:59:59.157465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:59:59.157465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:59:59.158523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:59.158523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:59.158523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:59.158523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:59.158523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:59.158523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:59.158523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:59.158523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:59.158523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:59.158523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:59.158523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:59.158523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:59.158523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:59.158523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:59.158523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:59.158523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:59.158523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:59:59.159413 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:59:59.159413 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:59.159413 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:59.159413 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 277/561] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:59.160178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634f044-1ba6-4607-9857-ecee608db5ec None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:59.160466 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634f044-1ba6-4607-9857-ecee608db5ec None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:59.160796 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634f044-1ba6-4607-9857-ecee608db5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:59.186133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-f634f044-1ba6-4607-9857-ecee608db5ec None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:59:59.187615 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-f634f044-1ba6-4607-9857-ecee608db5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:59:59.239801 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:59.244078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:59.276384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987eb8c-83da-45b6-a807-b24d064791d6 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:59.277810 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 278/562] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 19:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 19:59:59.282695 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634f044-1ba6-4607-9857-ecee608db5ec None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:59.317475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb86fa3-ab69-4e56-88f6-c1a2be4813a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:59.318182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb86fa3-ab69-4e56-88f6-c1a2be4813a0 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:59.318529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb86fa3-ab69-4e56-88f6-c1a2be4813a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:59.318926 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb86fa3-ab69-4e56-88f6-c1a2be4813a0 None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:59.319750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb86fa3-ab69-4e56-88f6-c1a2be4813a0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:59.320057 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb86fa3-ab69-4e56-88f6-c1a2be4813a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:59.321574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb86fa3-ab69-4e56-88f6-c1a2be4813a0 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:59.341276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 285/563] 10.222.0.73 () {56 vars in 1133 bytes} [Mon Aug 29 19:59:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:59.342250 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb86fa3-ab69-4e56-88f6-c1a2be4813a0 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:59.423248 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634f044-1ba6-4607-9857-ecee608db5ec None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:59.460484 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb86fa3-ab69-4e56-88f6-c1a2be4813a0 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:59.468126 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097a2d4-2e5a-4a6f-badd-e246177f9486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:59.469029 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097a2d4-2e5a-4a6f-badd-e246177f9486 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:59.469956 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097a2d4-2e5a-4a6f-badd-e246177f9486 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:59.470396 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097a2d4-2e5a-4a6f-badd-e246177f9486 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:59.471181 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097a2d4-2e5a-4a6f-badd-e246177f9486 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:59.471961 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097a2d4-2e5a-4a6f-badd-e246177f9486 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:59.474094 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097a2d4-2e5a-4a6f-badd-e246177f9486 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:59.489879 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 279/564] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 19:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:59.491582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097a2d4-2e5a-4a6f-badd-e246177f9486 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:59.493602 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:59.501014 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2d453-1032-48dc-91e7-71c0630fa773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:59.502034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2d453-1032-48dc-91e7-71c0630fa773 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:59.502523 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2d453-1032-48dc-91e7-71c0630fa773 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:59.503102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2d453-1032-48dc-91e7-71c0630fa773 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:59.514675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc2d453-1032-48dc-91e7-71c0630fa773 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:59.520129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-3dc2d453-1032-48dc-91e7-71c0630fa773 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 19:59:59.520129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:59.520129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:59.520129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:59.520129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:59.520129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:59.520129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:59.520129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:59.520129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:59.520129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:59.520129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:59.520129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:59.520129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:59.520129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:59.520129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:59.520129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:59.521055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:59.521055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:59.521055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:59.521055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:59.521055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:59.521055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:59.521055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:59.521055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:59.521055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:59.521055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:59.521055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:59.521055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:59.521055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:59.521055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:59.521055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:59.521055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:59.521930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:59.521930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:59.521930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:59.521930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:59.521930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:59.521930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:59.521930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:59.521930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:59.521930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:59.521930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:59.521930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 19:59:59.521930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:59.524266 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 286/565] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 19:59:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:59.525321 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2d453-1032-48dc-91e7-71c0630fa773 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:59.527440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:59.615155 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097a2d4-2e5a-4a6f-badd-e246177f9486 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:59.639764 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2d453-1032-48dc-91e7-71c0630fa773 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:59.654097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766714b6-ea92-40e6-80a0-f89f2ca7e5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:59.655236 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766714b6-ea92-40e6-80a0-f89f2ca7e5bc None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:59.655523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766714b6-ea92-40e6-80a0-f89f2ca7e5bc None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:59.656068 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766714b6-ea92-40e6-80a0-f89f2ca7e5bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:59.669037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766714b6-ea92-40e6-80a0-f89f2ca7e5bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:59.670856 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766714b6-ea92-40e6-80a0-f89f2ca7e5bc None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:59.677571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c7afd-e70f-402f-b5d3-d960226f336a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:59.678204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c7afd-e70f-402f-b5d3-d960226f336a None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:59.678529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c7afd-e70f-402f-b5d3-d960226f336a None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:59.679236 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c7afd-e70f-402f-b5d3-d960226f336a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:59.686715 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c7afd-e70f-402f-b5d3-d960226f336a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:59.687109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-766714b6-ea92-40e6-80a0-f89f2ca7e5bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:59.687109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:59.687109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:59.687109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:59.687109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:59.687109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:59.687109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:59.687109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:59.687109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:59.687109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:59.687109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:59.687109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:59.687109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 19:59:59.687109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 19:59:59.687109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 19:59:59.687109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 19:59:59.688240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:59.688240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:59.688240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:59.688240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:59.688240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:59.688240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:59.688240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:59.688240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:59.688240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:59.688240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:59.688240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:59.688240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:59.688240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:59.688240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:59.688240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:59.688240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:59.688240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 19:59:59.689106 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 19:59:59.689106 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 19:59:59.689106 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 19:59:59.689271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c7afd-e70f-402f-b5d3-d960226f336a None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:59.692846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 280/566] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 19:59:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:59.693688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766714b6-ea92-40e6-80a0-f89f2ca7e5bc None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:59.702421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-5a5c7afd-e70f-402f-b5d3-d960226f336a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 19:59:59.702421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:59.702421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:59.702421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:59.702421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:59.702421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:59.702421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:59.702421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:59.702421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:59.702421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:59.702421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:59.702421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:59.702421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 19:59:59.702421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 19:59:59.702421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 19:59:59.703329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 19:59:59.703329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:59.703329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:59.703329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:59.703329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:59.703329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:59.703329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:59.703329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:59.703329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:59.703329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:59.703329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:59.703329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:59.703329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:59.703329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:59.703329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:59.703329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:59.703329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:59.704211 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 19:59:59.704211 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 19:59:59.704211 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 19:59:59.704211 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 19:59:59.704211 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 19:59:59.704211 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 19:59:59.704211 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 19:59:59.704211 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:59.704211 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 287/567] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 19:59:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:59.705642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c7afd-e70f-402f-b5d3-d960226f336a None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:59.760925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:59.791225 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:59:59.807178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766714b6-ea92-40e6-80a0-f89f2ca7e5bc None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:59.810524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c7afd-e70f-402f-b5d3-d960226f336a None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:59.844452 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e5b0c-39e8-4d52-b35f-6f2d63548b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:59.844734 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da94d8a-9468-4e48-aa36-0a4e8a25f245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:59:59.845065 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e5b0c-39e8-4d52-b35f-6f2d63548b90 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:59.845346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e5b0c-39e8-4d52-b35f-6f2d63548b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:59.845440 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da94d8a-9468-4e48-aa36-0a4e8a25f245 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:59:59.845624 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e5b0c-39e8-4d52-b35f-6f2d63548b90 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:59.845757 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da94d8a-9468-4e48-aa36-0a4e8a25f245 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:59:59.845937 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da94d8a-9468-4e48-aa36-0a4e8a25f245 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:59:59.846423 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e5b0c-39e8-4d52-b35f-6f2d63548b90 None admin] RBAC: Adding query filter params (name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:59:59.846761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e5b0c-39e8-4d52-b35f-6f2d63548b90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:59:59.848128 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e5b0c-39e8-4d52-b35f-6f2d63548b90 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:59:59.856430 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 281/568] 10.222.0.73 () {56 vars in 1103 bytes} [Mon Aug 29 19:59:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:59:59.857199 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e5b0c-39e8-4d52-b35f-6f2d63548b90 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:59.859060 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da94d8a-9468-4e48-aa36-0a4e8a25f245 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 19:59:59.865149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-4da94d8a-9468-4e48-aa36-0a4e8a25f245 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 19:59:59.865149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 19:59:59.865149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 19:59:59.865149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 19:59:59.865149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 19:59:59.865149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 19:59:59.865149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 19:59:59.865149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 19:59:59.865149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 19:59:59.865149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 19:59:59.865149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 19:59:59.865149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 19:59:59.865149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 19:59:59.865149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 19:59:59.865149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 19:59:59.865149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 19:59:59.866393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 19:59:59.866393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 19:59:59.866393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 19:59:59.866393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 19:59:59.866393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 19:59:59.866393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 19:59:59.866393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 19:59:59.866393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 19:59:59.866393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 19:59:59.866393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 19:59:59.866393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 19:59:59.866393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 19:59:59.866393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 19:59:59.866393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 19:59:59.866393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 19:59:59.866393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 19:59:59.867544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 19:59:59.867544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 19:59:59.867544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 19:59:59.867544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 19:59:59.867544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 19:59:59.867544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 19:59:59.867544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 19:59:59.867544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 19:59:59.867544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 19:59:59.867544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 19:59:59.867544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 19:59:59.867544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 19:59:59.867544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 288/569] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 19:59:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 19:59:59.867544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da94d8a-9468-4e48-aa36-0a4e8a25f245 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:59:59.966958 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e5b0c-39e8-4d52-b35f-6f2d63548b90 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:59.970212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da94d8a-9468-4e48-aa36-0a4e8a25f245 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:59:59.983072 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:00.000651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c52389-00d5-4ec4-9291-529b1fdf8b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:00.001251 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c52389-00d5-4ec4-9291-529b1fdf8b16 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:00.001532 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c52389-00d5-4ec4-9291-529b1fdf8b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:00.001875 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c52389-00d5-4ec4-9291-529b1fdf8b16 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:00.002502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c52389-00d5-4ec4-9291-529b1fdf8b16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:00.002804 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c52389-00d5-4ec4-9291-529b1fdf8b16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:00.004208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c52389-00d5-4ec4-9291-529b1fdf8b16 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:00.005342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73d1190-5492-4f70-9ea4-c43b4ca8a8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:00.006037 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d1190-5492-4f70-9ea4-c43b4ca8a8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:00.006381 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d1190-5492-4f70-9ea4-c43b4ca8a8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:00.006759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d1190-5492-4f70-9ea4-c43b4ca8a8b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:00.007570 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73d1190-5492-4f70-9ea4-c43b4ca8a8b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:00.007877 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73d1190-5492-4f70-9ea4-c43b4ca8a8b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:00.009344 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73d1190-5492-4f70-9ea4-c43b4ca8a8b5 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:00.021784 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 282/570] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 19:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:00.023156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06e9b0-688b-48b1-b054-aec777606d24 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:00.023444 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06e9b0-688b-48b1-b054-aec777606d24 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:00.023739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06e9b0-688b-48b1-b054-aec777606d24 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:00.028940 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 289/571] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 19:59:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:00.029098 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 283/572] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:00.029174 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73d1190-5492-4f70-9ea4-c43b4ca8a8b5 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:00.029243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06e9b0-688b-48b1-b054-aec777606d24 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:00.036935 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:00.129984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73d1190-5492-4f70-9ea4-c43b4ca8a8b5 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:00.135132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06e9b0-688b-48b1-b054-aec777606d24 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:00.164009 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f58c0-695f-4091-abc5-37e8e0dafe13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:00.164639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f58c0-695f-4091-abc5-37e8e0dafe13 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:00.165018 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f58c0-695f-4091-abc5-37e8e0dafe13 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:00.165181 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f58c0-695f-4091-abc5-37e8e0dafe13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:00.169151 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06920e3c-d4de-45f7-985f-b678243ef4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:00.169779 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06920e3c-d4de-45f7-985f-b678243ef4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:00.170063 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06920e3c-d4de-45f7-985f-b678243ef4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:00.170338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06920e3c-d4de-45f7-985f-b678243ef4d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:00.171002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06920e3c-d4de-45f7-985f-b678243ef4d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:00.171316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06920e3c-d4de-45f7-985f-b678243ef4d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:00.172668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06920e3c-d4de-45f7-985f-b678243ef4d2 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:00.175610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f58c0-695f-4091-abc5-37e8e0dafe13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:00.177053 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f58c0-695f-4091-abc5-37e8e0dafe13 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:00.180675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 284/573] 10.222.0.73 () {56 vars in 1103 bytes} [Mon Aug 29 20:00:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:00.181483 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06920e3c-d4de-45f7-985f-b678243ef4d2 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:00.193436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-827f58c0-695f-4091-abc5-37e8e0dafe13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:00.193436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:00.193436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:00.193436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:00.193436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:00.193436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:00.193436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:00.193436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:00.193436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:00.193436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:00.193436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:00.193436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:00.193436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:00.193436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:00.193436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:00.193436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:00.194451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:00.194451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:00.194451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:00.194451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:00.194451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:00.194451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:00.194451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:00.194451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:00.194451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:00.194451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:00.194451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:00.194451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:00.194451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:00.194451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:00.194451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:00.194451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:00.194451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:00.195329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:00.195329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:00.195329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:00.195329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 290/574] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:00.195329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f58c0-695f-4091-abc5-37e8e0dafe13 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:00.221072 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:00.255115 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:00.289633 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06920e3c-d4de-45f7-985f-b678243ef4d2 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:00.299648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f58c0-695f-4091-abc5-37e8e0dafe13 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:00.323487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf3594-e2e7-42e0-9b65-4efcb6f64c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:00.324115 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf3594-e2e7-42e0-9b65-4efcb6f64c0a None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:00.324402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf3594-e2e7-42e0-9b65-4efcb6f64c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:00.324676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf3594-e2e7-42e0-9b65-4efcb6f64c0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:00.332499 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf3594-e2e7-42e0-9b65-4efcb6f64c0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:00.333967 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf3594-e2e7-42e0-9b65-4efcb6f64c0a None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:00.336030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8018e6-759b-4acf-a5a0-4333c9bb589f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:00.336672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8018e6-759b-4acf-a5a0-4333c9bb589f None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:00.336954 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8018e6-759b-4acf-a5a0-4333c9bb589f None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:00.337230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8018e6-759b-4acf-a5a0-4333c9bb589f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:00.337980 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8018e6-759b-4acf-a5a0-4333c9bb589f None admin] RBAC: Adding query filter params (role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=b92e30da8f394f3abc19b9e86ef126e5) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:00.338290 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8018e6-759b-4acf-a5a0-4333c9bb589f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:00.339702 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8018e6-759b-4acf-a5a0-4333c9bb589f None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:00.345219 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 291/575] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:00:00 2022] GET /identity/v3/role_assignments?user.id=b92e30da8f394f3abc19b9e86ef126e5&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:00.345675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-1dbf3594-e2e7-42e0-9b65-4efcb6f64c0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:00.345675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:00.345675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:00.345675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:00.345675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:00.345675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:00.345675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:00.345675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:00.345675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:00.345675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:00.345675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:00.345675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:00.345675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:00.345675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:00.345675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:00.346599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:00.346599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:00.346599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:00.346599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:00.346599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:00.346599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:00.346599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:00.346599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:00.346599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:00.346599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:00.346599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:00.346599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:00.346599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:00.346599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:00.346599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:00.346599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:00.346599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:00.347552 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:00.347552 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:00.347552 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:00.347552 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:00.347552 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:00.347552 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:00.347552 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:00.347552 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:00.347972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380cee30-74be-4e2c-87a2-d8f25909eed5 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:00.347972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380cee30-74be-4e2c-87a2-d8f25909eed5 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:00.347972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380cee30-74be-4e2c-87a2-d8f25909eed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:00.349653 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 285/576] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:00.350971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf3594-e2e7-42e0-9b65-4efcb6f64c0a None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:00.374058 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-380cee30-74be-4e2c-87a2-d8f25909eed5 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:00.375316 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-380cee30-74be-4e2c-87a2-d8f25909eed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:00.457000 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 292/577] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:00:00.458141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380cee30-74be-4e2c-87a2-d8f25909eed5 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:00.482147 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf3594-e2e7-42e0-9b65-4efcb6f64c0a None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:00.482880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:00.516604 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:00.527621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f34c4-7d26-4af4-b87f-415f0bd9c8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:00.529080 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f34c4-7d26-4af4-b87f-415f0bd9c8cc None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:00.529567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f34c4-7d26-4af4-b87f-415f0bd9c8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:00.530035 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f34c4-7d26-4af4-b87f-415f0bd9c8cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:00.541856 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f34c4-7d26-4af4-b87f-415f0bd9c8cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:00.544555 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f34c4-7d26-4af4-b87f-415f0bd9c8cc None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:00.560037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-f81f34c4-7d26-4af4-b87f-415f0bd9c8cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:00.560037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:00.560037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:00.560037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:00.560037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:00.560037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:00.560037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:00.560037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:00.560037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:00.560037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:00.560037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:00.560037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:00.560037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:00.560037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:00.560037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:00.560037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:00.561006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:00.561006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:00.561006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:00.561006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:00.561006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:00.561006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:00.561006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:00.561006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:00.561006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:00.561006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:00.561006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:00.561006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:00.561006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:00.561006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:00.561006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:00.561006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:00.561006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:00.561915 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:00.561915 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:00.561915 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:00.563728 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 286/578] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:00.564725 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f34c4-7d26-4af4-b87f-415f0bd9c8cc None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:00.578616 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380cee30-74be-4e2c-87a2-d8f25909eed5 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:00.630055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296ef35-c9ba-4248-97a3-2f55233ee4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:00.631101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296ef35-c9ba-4248-97a3-2f55233ee4c9 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:00.631591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296ef35-c9ba-4248-97a3-2f55233ee4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:00.632090 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296ef35-c9ba-4248-97a3-2f55233ee4c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:00.632926 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296ef35-c9ba-4248-97a3-2f55233ee4c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:00.633434 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296ef35-c9ba-4248-97a3-2f55233ee4c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:00.635030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296ef35-c9ba-4248-97a3-2f55233ee4c9 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:00.649070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 293/579] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:00.650355 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296ef35-c9ba-4248-97a3-2f55233ee4c9 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:00.699144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f34c4-7d26-4af4-b87f-415f0bd9c8cc None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:00.741205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f05f1-b983-44ff-89f8-6db49f96ad75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:00.742213 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f05f1-b983-44ff-89f8-6db49f96ad75 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:00.742921 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f05f1-b983-44ff-89f8-6db49f96ad75 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:00.743393 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f05f1-b983-44ff-89f8-6db49f96ad75 None admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:00.744344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f05f1-b983-44ff-89f8-6db49f96ad75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:00.744857 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f05f1-b983-44ff-89f8-6db49f96ad75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:00.746500 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f05f1-b983-44ff-89f8-6db49f96ad75 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:00.761636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 287/580] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:00:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:00.761852 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:00.763781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f05f1-b983-44ff-89f8-6db49f96ad75 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:00.789772 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296ef35-c9ba-4248-97a3-2f55233ee4c9 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:00.816730 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:00.828660 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2008ab-b184-41ff-8b0c-bbe3033c0264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:00.829722 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2008ab-b184-41ff-8b0c-bbe3033c0264 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:00.830101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2008ab-b184-41ff-8b0c-bbe3033c0264 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:00.830448 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2008ab-b184-41ff-8b0c-bbe3033c0264 None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:00.831095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2008ab-b184-41ff-8b0c-bbe3033c0264 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:00.831396 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2008ab-b184-41ff-8b0c-bbe3033c0264 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:00.832774 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2008ab-b184-41ff-8b0c-bbe3033c0264 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:00.846859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 294/581] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:00.847959 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2008ab-b184-41ff-8b0c-bbe3033c0264 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:00.888379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f05f1-b983-44ff-89f8-6db49f96ad75 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:00.925914 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c009be2-05e2-42b5-b732-a2312c505a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:00.926861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c009be2-05e2-42b5-b732-a2312c505a32 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:00.927143 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c009be2-05e2-42b5-b732-a2312c505a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:00.927415 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c009be2-05e2-42b5-b732-a2312c505a32 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:00.935227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c009be2-05e2-42b5-b732-a2312c505a32 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:00.936661 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c009be2-05e2-42b5-b732-a2312c505a32 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:00.948894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-0c009be2-05e2-42b5-b732-a2312c505a32 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 20:00:00.948894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:00.948894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:00.948894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:00.948894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:00.948894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:00.948894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:00.948894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:00.948894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:00.948894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:00.948894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:00.948894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:00.948894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:00:00.948894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:00:00.948894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:00:00.948894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:00:00.949986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:00.949986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:00.949986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:00.949986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:00.949986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:00.949986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:00.949986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:00.949986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:00.949986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:00.949986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:00.949986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:00.949986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:00.949986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:00.949986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:00.949986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:00.949986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:00.949986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:00.949986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:00.950978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:00.950978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:00.950978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:00:00.950978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:00:00.950978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:00:00.950978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:00:00.950978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:00:00.950978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:00:00.950978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 20:00:00.950978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:00.954535 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 288/582] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 20:00:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:00.955563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c009be2-05e2-42b5-b732-a2312c505a32 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:00.968259 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2008ab-b184-41ff-8b0c-bbe3033c0264 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:01.006472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea5c2e4-c6db-486f-90a7-53291287ee32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:01.007387 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5c2e4-c6db-486f-90a7-53291287ee32 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:01.007677 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5c2e4-c6db-486f-90a7-53291287ee32 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:01.007941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5c2e4-c6db-486f-90a7-53291287ee32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:01.015582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:01.019753 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea5c2e4-c6db-486f-90a7-53291287ee32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:00:01.024491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-8ea5c2e4-c6db-486f-90a7-53291287ee32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:00:01.024491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:01.024491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:01.024491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:01.024491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:01.024491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:01.024491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:01.024491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:01.024491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:01.024491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:01.024491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:01.024491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:01.024491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:00:01.024491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:00:01.024491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:00:01.024491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:00:01.025619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:00:01.025619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:00:01.025619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:00:01.025619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:00:01.025619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:01.025619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:01.025619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:01.025619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:01.025619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:01.025619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:01.025619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:01.025619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:01.025619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:01.025619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:01.025619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:01.025619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:01.026676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:01.026676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:01.026676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:01.026676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:01.026676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:00:01.026676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:00:01.026676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:00:01.026676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:00:01.026676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:00:01.026676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:00:01.026676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:00:01.026676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:01.030196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 295/583] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 20:00:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:01.030685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea5c2e4-c6db-486f-90a7-53291287ee32 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:01.078442 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:01.084877 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c009be2-05e2-42b5-b732-a2312c505a32 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:01.128165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a50f5f-1f8c-4c39-b564-1ecc5ccd893c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:01.128940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a50f5f-1f8c-4c39-b564-1ecc5ccd893c None admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:01.129296 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a50f5f-1f8c-4c39-b564-1ecc5ccd893c None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:01.129642 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a50f5f-1f8c-4c39-b564-1ecc5ccd893c None admin] PATH_INFO: `/v3/projects/2f5ec615d98548159487c97cead336a0/users/95cf77a92a174ff8b0503baf5c7e63be/roles/8ca2ec70f49a4f4da3ce9f3ac0b73083` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:01.142298 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea5c2e4-c6db-486f-90a7-53291287ee32 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:01.145579 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a50f5f-1f8c-4c39-b564-1ecc5ccd893c None admin] RBAC: Authorizing `identity:create_grant(project_id=2f5ec615d98548159487c97cead336a0, user_id=95cf77a92a174ff8b0503baf5c7e63be, role_id=8ca2ec70f49a4f4da3ce9f3ac0b73083)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:01.147171 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a50f5f-1f8c-4c39-b564-1ecc5ccd893c None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:01.169865 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 289/584] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 20:00:00 2022] PUT /identity/v3/projects/2f5ec615d98548159487c97cead336a0/users/95cf77a92a174ff8b0503baf5c7e63be/roles/8ca2ec70f49a4f4da3ce9f3ac0b73083 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:00:01.169865 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a50f5f-1f8c-4c39-b564-1ecc5ccd893c None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:01.201280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5c296-b216-44ad-bbca-542e69923156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:01.202318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5c296-b216-44ad-bbca-542e69923156 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:01.202655 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5c296-b216-44ad-bbca-542e69923156 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:01.202936 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5c296-b216-44ad-bbca-542e69923156 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:01.213953 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5c296-b216-44ad-bbca-542e69923156 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:01.215521 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5c296-b216-44ad-bbca-542e69923156 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:01.238817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-c8f5c296-b216-44ad-bbca-542e69923156 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 20:00:01.238817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:01.238817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:01.238817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:01.238817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:01.238817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:01.238817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:01.238817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:01.238817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:01.238817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:01.238817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:01.238817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:01.238817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:00:01.238817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:00:01.238817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:00:01.238817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:00:01.239819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:01.239819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:01.239819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:01.239819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:01.239819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:01.239819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:01.239819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:01.239819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:01.239819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:01.239819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:01.239819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:01.239819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:01.239819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:01.239819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:01.239819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:01.239819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:01.239819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:01.239819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:01.240750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:01.240750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:01.240750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:00:01.240750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:00:01.240750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:00:01.240750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:00:01.240750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:00:01.240750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:00:01.240750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 20:00:01.240750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:01.247885 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 296/585] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 20:00:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:01.247885 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5c296-b216-44ad-bbca-542e69923156 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:01.301531 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a50f5f-1f8c-4c39-b564-1ecc5ccd893c None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:01.336958 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:01.349671 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d68465-fbc9-4216-9d3f-84768b8af196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:01.351149 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d68465-fbc9-4216-9d3f-84768b8af196 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:01.351431 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d68465-fbc9-4216-9d3f-84768b8af196 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:01.351709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d68465-fbc9-4216-9d3f-84768b8af196 None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:01.352462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d68465-fbc9-4216-9d3f-84768b8af196 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:01.352773 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d68465-fbc9-4216-9d3f-84768b8af196 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:01.361747 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d68465-fbc9-4216-9d3f-84768b8af196 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:01.380751 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 290/586] 10.222.0.73 () {56 vars in 1137 bytes} [Mon Aug 29 20:00:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:01.381623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d68465-fbc9-4216-9d3f-84768b8af196 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:01.397408 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:01.402351 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5c296-b216-44ad-bbca-542e69923156 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:01.453045 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972b5d7-6419-4961-bd9f-16a485712f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:01.453726 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972b5d7-6419-4961-bd9f-16a485712f9c None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:01.454512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972b5d7-6419-4961-bd9f-16a485712f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:01.454785 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972b5d7-6419-4961-bd9f-16a485712f9c None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:01.455793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972b5d7-6419-4961-bd9f-16a485712f9c None admin] RBAC: Adding query filter params (name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:01.456102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972b5d7-6419-4961-bd9f-16a485712f9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:01.458389 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972b5d7-6419-4961-bd9f-16a485712f9c None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:01.471425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 297/587] 10.222.0.73 () {56 vars in 1103 bytes} [Mon Aug 29 20:00:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:01.472282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972b5d7-6419-4961-bd9f-16a485712f9c None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:01.506558 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d68465-fbc9-4216-9d3f-84768b8af196 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:01.554004 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe649031-e592-4dc1-9fdf-819d6688aac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:01.555404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe649031-e592-4dc1-9fdf-819d6688aac1 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:01.555687 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe649031-e592-4dc1-9fdf-819d6688aac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:01.555959 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe649031-e592-4dc1-9fdf-819d6688aac1 None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:01.556690 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe649031-e592-4dc1-9fdf-819d6688aac1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:01.556994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe649031-e592-4dc1-9fdf-819d6688aac1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:01.558435 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe649031-e592-4dc1-9fdf-819d6688aac1 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:01.585630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 291/588] 10.222.0.73 () {56 vars in 1137 bytes} [Mon Aug 29 20:00:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:01.586742 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe649031-e592-4dc1-9fdf-819d6688aac1 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:01.587771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972b5d7-6419-4961-bd9f-16a485712f9c None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:01.615625 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:01.632447 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473406ba-5e45-4c58-918e-a0c1468ff023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:01.633538 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473406ba-5e45-4c58-918e-a0c1468ff023 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:01.634142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473406ba-5e45-4c58-918e-a0c1468ff023 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:01.634679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473406ba-5e45-4c58-918e-a0c1468ff023 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:01.647726 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473406ba-5e45-4c58-918e-a0c1468ff023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:01.657758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473406ba-5e45-4c58-918e-a0c1468ff023 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:01.668202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-473406ba-5e45-4c58-918e-a0c1468ff023 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:01.668202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:01.668202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:01.668202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:01.668202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:01.668202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:01.668202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:01.668202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:01.668202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:01.668202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:01.668202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:01.668202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:01.668202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:01.668202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:01.668202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:01.668202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:01.669252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:01.669252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:01.669252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:01.669252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:01.669252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:01.669252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:01.669252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:01.669252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:01.669252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:01.669252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:01.669252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:01.669252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:01.669252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:01.669252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:01.669252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:01.669252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:01.669252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:01.670171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:01.670171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:01.670171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:01.674107 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 298/589] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:01.674905 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:01.675504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473406ba-5e45-4c58-918e-a0c1468ff023 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:01.722927 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe649031-e592-4dc1-9fdf-819d6688aac1 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:01.773727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dec5ad-ee4b-4000-afbe-d5bcbc600755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:01.774387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dec5ad-ee4b-4000-afbe-d5bcbc600755 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:01.774666 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dec5ad-ee4b-4000-afbe-d5bcbc600755 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:01.774957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dec5ad-ee4b-4000-afbe-d5bcbc600755 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:01.789700 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dec5ad-ee4b-4000-afbe-d5bcbc600755 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:01.791937 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dec5ad-ee4b-4000-afbe-d5bcbc600755 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:01.794684 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473406ba-5e45-4c58-918e-a0c1468ff023 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:01.813170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-00dec5ad-ee4b-4000-afbe-d5bcbc600755 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:01.813170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:01.813170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:01.813170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:01.813170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:01.813170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:01.813170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:01.813170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:01.813170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:01.813170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:01.813170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:01.813170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:01.813170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:01.813170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:01.813170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:01.813170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:01.814157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:01.814157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:01.814157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:01.814157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:01.814157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:01.814157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:01.814157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:01.814157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:01.814157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:01.814157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:01.814157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:01.814157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:01.814157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:01.814157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:01.814157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:01.814157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:01.814157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:01.815051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:01.815051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:01.815051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:01.817793 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 292/590] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:01.818839 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dec5ad-ee4b-4000-afbe-d5bcbc600755 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:01.843758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499885c1-f258-4dd8-883d-d8b71e069675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:01.844419 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499885c1-f258-4dd8-883d-d8b71e069675 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:01.844699 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499885c1-f258-4dd8-883d-d8b71e069675 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:01.844974 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499885c1-f258-4dd8-883d-d8b71e069675 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:01.861064 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499885c1-f258-4dd8-883d-d8b71e069675 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:01.862726 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499885c1-f258-4dd8-883d-d8b71e069675 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:01.880246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-499885c1-f258-4dd8-883d-d8b71e069675 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:01.880246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:01.880246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:01.880246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:01.880246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:01.880246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:01.880246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:01.880246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:01.880246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:01.880246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:01.880246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:01.880246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:01.880246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:01.880246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:01.880246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:01.880246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:01.881281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:01.881281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:01.881281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:01.881281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:01.881281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:01.881281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:01.881281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:01.881281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:01.881281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:01.881281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:01.881281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:01.881281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:01.881281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:01.881281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:01.881281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:01.881281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:01.881281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:01.882208 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:01.882208 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:01.882208 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:01.882208 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 299/591] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:01.882936 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499885c1-f258-4dd8-883d-d8b71e069675 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:01.910901 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:01.939905 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dec5ad-ee4b-4000-afbe-d5bcbc600755 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:01.958790 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:01.978331 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a38bbd-c5f7-469b-8480-a861f41075f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:01.979242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a38bbd-c5f7-469b-8480-a861f41075f7 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:01.979674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a38bbd-c5f7-469b-8480-a861f41075f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:01.980091 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a38bbd-c5f7-469b-8480-a861f41075f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:01.980879 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a38bbd-c5f7-469b-8480-a861f41075f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:01.981328 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a38bbd-c5f7-469b-8480-a861f41075f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:01.983235 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a38bbd-c5f7-469b-8480-a861f41075f7 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:01.994478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499885c1-f258-4dd8-883d-d8b71e069675 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:02.001623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 293/592] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:02.002720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a38bbd-c5f7-469b-8480-a861f41075f7 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:02.032295 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad94db3-2d05-410e-a96b-7cae19b4c1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:02.032775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad94db3-2d05-410e-a96b-7cae19b4c1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:02.033057 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad94db3-2d05-410e-a96b-7cae19b4c1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:02.033333 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad94db3-2d05-410e-a96b-7cae19b4c1f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:02.034001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad94db3-2d05-410e-a96b-7cae19b4c1f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:02.034330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad94db3-2d05-410e-a96b-7cae19b4c1f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:02.035759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad94db3-2d05-410e-a96b-7cae19b4c1f5 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:02.053856 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 300/593] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:02.054834 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad94db3-2d05-410e-a96b-7cae19b4c1f5 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:02.120612 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a38bbd-c5f7-469b-8480-a861f41075f7 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:02.152565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:02.159489 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cac2ec-0b45-48db-b089-80f97ba5af20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:02.160422 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cac2ec-0b45-48db-b089-80f97ba5af20 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:02.160422 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cac2ec-0b45-48db-b089-80f97ba5af20 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:02.160636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cac2ec-0b45-48db-b089-80f97ba5af20 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:02.161259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cac2ec-0b45-48db-b089-80f97ba5af20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:02.161563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cac2ec-0b45-48db-b089-80f97ba5af20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:02.162985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cac2ec-0b45-48db-b089-80f97ba5af20 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:02.174092 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad94db3-2d05-410e-a96b-7cae19b4c1f5 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:02.175674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 294/594] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:02.176526 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cac2ec-0b45-48db-b089-80f97ba5af20 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:02.212165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757ed92-dd6e-4645-acbe-fda50b82fa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:02.213104 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757ed92-dd6e-4645-acbe-fda50b82fa96 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:02.213554 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757ed92-dd6e-4645-acbe-fda50b82fa96 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:02.214009 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757ed92-dd6e-4645-acbe-fda50b82fa96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:02.221474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:02.223012 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757ed92-dd6e-4645-acbe-fda50b82fa96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:02.225752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757ed92-dd6e-4645-acbe-fda50b82fa96 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:02.239224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-3757ed92-dd6e-4645-acbe-fda50b82fa96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:02.239224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:02.239224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:02.239224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:02.239224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:02.239224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:02.239224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:02.239224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:02.239224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:02.239224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:02.239224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:02.239224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:02.239224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:02.239224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:02.239224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:02.240338 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:02.240338 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:02.240338 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:02.240338 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:02.240338 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:02.240338 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:02.240338 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:02.240338 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:02.240338 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:02.240338 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:02.240338 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:02.240338 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:02.240338 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:02.240338 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:02.240338 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:02.240338 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:02.240338 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:02.241332 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:02.241332 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:02.241332 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:02.241332 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:02.241332 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:02.241332 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:02.241332 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:02.241332 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:02.243516 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 301/595] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:02.244419 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757ed92-dd6e-4645-acbe-fda50b82fa96 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:02.301948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cac2ec-0b45-48db-b089-80f97ba5af20 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:02.341446 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095e6e1-e89c-4fe4-807b-08815234eea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:02.342272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095e6e1-e89c-4fe4-807b-08815234eea8 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:02.342552 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095e6e1-e89c-4fe4-807b-08815234eea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:02.342824 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095e6e1-e89c-4fe4-807b-08815234eea8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:02.350898 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c095e6e1-e89c-4fe4-807b-08815234eea8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:02.352320 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c095e6e1-e89c-4fe4-807b-08815234eea8 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:02.364600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-c095e6e1-e89c-4fe4-807b-08815234eea8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 20:00:02.364600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:02.364600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:02.364600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:02.364600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:02.364600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:02.364600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:02.364600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:02.364600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:02.364600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:02.364600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:02.364600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:02.364600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:00:02.364600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:00:02.364600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:00:02.364600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:00:02.365547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:02.365547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:02.365547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:02.365547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:02.365547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:02.365547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:02.365547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:02.365547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:02.365547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:02.365547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:02.365547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:02.365547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:02.365547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:02.365547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:02.365547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:02.365547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:02.365547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:02.365547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:02.366564 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:02.366564 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:02.366564 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:00:02.366564 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:00:02.366564 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:00:02.366564 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:00:02.366564 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:00:02.366564 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:00:02.366564 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 20:00:02.366564 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:02.366564 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 295/596] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 20:00:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:02.366564 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095e6e1-e89c-4fe4-807b-08815234eea8 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:02.382820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757ed92-dd6e-4645-acbe-fda50b82fa96 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:02.421849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb24558-d0ab-42f1-aab2-920267af693d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:02.422710 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb24558-d0ab-42f1-aab2-920267af693d None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:02.423153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb24558-d0ab-42f1-aab2-920267af693d None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:02.423962 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb24558-d0ab-42f1-aab2-920267af693d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:02.432978 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb24558-d0ab-42f1-aab2-920267af693d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:02.435569 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb24558-d0ab-42f1-aab2-920267af693d None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:02.448095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-6eb24558-d0ab-42f1-aab2-920267af693d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:02.448095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:02.448095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:02.448095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:02.448095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:02.448095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:02.448095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:02.448095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:02.448095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:02.448095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:02.448095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:02.448095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:02.448095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:02.448095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:02.448095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:02.449086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:02.449086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:02.449086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:02.449086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:02.449086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:02.449086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:02.449086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:02.449086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:02.449086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:02.449086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:02.449086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:02.449086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:02.449086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:02.449086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:02.449086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:02.449086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:02.449086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:02.451375 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:02.451375 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:02.451375 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:02.451375 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:02.451375 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:02.451375 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:02.451375 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:02.451375 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:02.454005 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 302/597] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:02.455159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb2be3-b8fb-434f-ae1b-9e737f865b3a None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:02.455442 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb2be3-b8fb-434f-ae1b-9e737f865b3a None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:02.455717 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb2be3-b8fb-434f-ae1b-9e737f865b3a None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:02.456988 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 303/598] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:02.457667 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb2be3-b8fb-434f-ae1b-9e737f865b3a None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:02.464427 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:02.477745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:02.482155 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095e6e1-e89c-4fe4-807b-08815234eea8 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:02.521345 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445436f-c344-4bdd-8c0c-d84d0e9c643c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:02.522127 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445436f-c344-4bdd-8c0c-d84d0e9c643c None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:02.522405 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445436f-c344-4bdd-8c0c-d84d0e9c643c None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:02.522675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445436f-c344-4bdd-8c0c-d84d0e9c643c None admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:02.523357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445436f-c344-4bdd-8c0c-d84d0e9c643c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:02.523659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445436f-c344-4bdd-8c0c-d84d0e9c643c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:02.525091 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445436f-c344-4bdd-8c0c-d84d0e9c643c None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:02.537573 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 296/599] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:00:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:02.538419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445436f-c344-4bdd-8c0c-d84d0e9c643c None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:02.578456 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb2be3-b8fb-434f-ae1b-9e737f865b3a None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:02.615040 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb1e3a-b797-4f44-99f1-f15ecb541bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:02.615877 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb1e3a-b797-4f44-99f1-f15ecb541bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:02.616301 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb1e3a-b797-4f44-99f1-f15ecb541bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:02.616715 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb1e3a-b797-4f44-99f1-f15ecb541bd8 None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:02.617580 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb1e3a-b797-4f44-99f1-f15ecb541bd8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:02.618070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb1e3a-b797-4f44-99f1-f15ecb541bd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:02.619746 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb1e3a-b797-4f44-99f1-f15ecb541bd8 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:02.640101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 304/600] 10.222.0.73 () {56 vars in 1139 bytes} [Mon Aug 29 20:00:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:02.641660 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bb035-2a31-4545-a018-8c78a2c805d2 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:02.641965 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bb035-2a31-4545-a018-8c78a2c805d2 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:02.642243 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bb035-2a31-4545-a018-8c78a2c805d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:02.649051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445436f-c344-4bdd-8c0c-d84d0e9c643c None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:02.661878 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-b37bb035-2a31-4545-a018-8c78a2c805d2 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:02.663074 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-b37bb035-2a31-4545-a018-8c78a2c805d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:02.693084 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d91b7-4fd6-4264-869c-7749fb22ccbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:02.693698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d91b7-4fd6-4264-869c-7749fb22ccbc None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:02.694001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d91b7-4fd6-4264-869c-7749fb22ccbc None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:02.694272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d91b7-4fd6-4264-869c-7749fb22ccbc None admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:02.694962 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d91b7-4fd6-4264-869c-7749fb22ccbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:02.695263 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d91b7-4fd6-4264-869c-7749fb22ccbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:02.696739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d91b7-4fd6-4264-869c-7749fb22ccbc None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:02.710229 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 297/601] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:00:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:02.711671 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d91b7-4fd6-4264-869c-7749fb22ccbc None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:02.733497 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:02.742397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:02.745427 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 305/602] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:00:02.746237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bb035-2a31-4545-a018-8c78a2c805d2 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:02.853610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bb035-2a31-4545-a018-8c78a2c805d2 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:02.865653 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d91b7-4fd6-4264-869c-7749fb22ccbc None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:02.894924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c892a-9c69-42b8-a5f3-2c11fd51b9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:02.895656 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c892a-9c69-42b8-a5f3-2c11fd51b9d8 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:02.895901 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c892a-9c69-42b8-a5f3-2c11fd51b9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:02.896176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c892a-9c69-42b8-a5f3-2c11fd51b9d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:02.907628 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c892a-9c69-42b8-a5f3-2c11fd51b9d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:02.909213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c892a-9c69-42b8-a5f3-2c11fd51b9d8 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:02.912751 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27efb3f-b8e6-4fad-a87a-c52508a8fe90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:02.913685 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27efb3f-b8e6-4fad-a87a-c52508a8fe90 None admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:02.914195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27efb3f-b8e6-4fad-a87a-c52508a8fe90 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:02.914670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27efb3f-b8e6-4fad-a87a-c52508a8fe90 None admin] PATH_INFO: `/v3/projects/2f5ec615d98548159487c97cead336a0/users/103b64b4b172467692d02851b2955b81/roles/8ca2ec70f49a4f4da3ce9f3ac0b73083` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:02.926137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-629c892a-9c69-42b8-a5f3-2c11fd51b9d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:02.926137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:02.926137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:02.926137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:02.926137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:02.926137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:02.926137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:02.926137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:02.926137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:02.926137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:02.926137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:02.926137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:02.926137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:02.926137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:02.926137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:02.926137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:02.927178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:02.927178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:02.927178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:02.927178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:02.927178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:02.927178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:02.927178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:02.927178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:02.927178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:02.927178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:02.927178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:02.927178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:02.927178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:02.927178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:02.927178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:02.927178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:02.927178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:02.928123 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:02.928123 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:02.928123 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:02.932023 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 306/603] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:02.932838 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c892a-9c69-42b8-a5f3-2c11fd51b9d8 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:02.936804 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27efb3f-b8e6-4fad-a87a-c52508a8fe90 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f5ec615d98548159487c97cead336a0, user_id=103b64b4b172467692d02851b2955b81, role_id=8ca2ec70f49a4f4da3ce9f3ac0b73083)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:02.938958 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27efb3f-b8e6-4fad-a87a-c52508a8fe90 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:02.962165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 298/604] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 20:00:02 2022] PUT /identity/v3/projects/2f5ec615d98548159487c97cead336a0/users/103b64b4b172467692d02851b2955b81/roles/8ca2ec70f49a4f4da3ce9f3ac0b73083 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:00:02.965920 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27efb3f-b8e6-4fad-a87a-c52508a8fe90 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:03.022807 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:03.086108 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:03.109376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c892a-9c69-42b8-a5f3-2c11fd51b9d8 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:03.134839 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27efb3f-b8e6-4fad-a87a-c52508a8fe90 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:03.154541 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a82872-7f85-4114-af7b-10ac10348c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:03.155088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a82872-7f85-4114-af7b-10ac10348c21 None admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:03.155315 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a82872-7f85-4114-af7b-10ac10348c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:03.155542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a82872-7f85-4114-af7b-10ac10348c21 None admin] PATH_INFO: `/v3/projects/2f5ec615d98548159487c97cead336a0/users/4a0b2ca8d14a47239a26912e58348f5a/roles/8ca2ec70f49a4f4da3ce9f3ac0b73083` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:03.174538 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a82872-7f85-4114-af7b-10ac10348c21 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f5ec615d98548159487c97cead336a0, user_id=4a0b2ca8d14a47239a26912e58348f5a, role_id=8ca2ec70f49a4f4da3ce9f3ac0b73083)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:03.175536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fba5c-6e51-4851-a9ca-686be9f2ca8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:03.176099 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a82872-7f85-4114-af7b-10ac10348c21 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:03.176210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fba5c-6e51-4851-a9ca-686be9f2ca8b None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:03.176414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fba5c-6e51-4851-a9ca-686be9f2ca8b None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:03.176680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fba5c-6e51-4851-a9ca-686be9f2ca8b None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:03.177322 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8fba5c-6e51-4851-a9ca-686be9f2ca8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:03.177617 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8fba5c-6e51-4851-a9ca-686be9f2ca8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:03.179384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8fba5c-6e51-4851-a9ca-686be9f2ca8b None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:03.193546 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 299/605] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:03.197657 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 307/606] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 20:00:02 2022] PUT /identity/v3/projects/2f5ec615d98548159487c97cead336a0/users/4a0b2ca8d14a47239a26912e58348f5a/roles/8ca2ec70f49a4f4da3ce9f3ac0b73083 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:00:03.197657 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6acb9-2bf7-4871-8131-976b46a7a711 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:03.197657 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6acb9-2bf7-4871-8131-976b46a7a711 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:03.197657 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6acb9-2bf7-4871-8131-976b46a7a711 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:03.197657 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 308/607] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:03.198187 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fba5c-6e51-4851-a9ca-686be9f2ca8b None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:03.201280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6acb9-2bf7-4871-8131-976b46a7a711 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:03.313425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:03.323400 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6acb9-2bf7-4871-8131-976b46a7a711 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:03.337125 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fba5c-6e51-4851-a9ca-686be9f2ca8b None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:03.365516 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85638620-76ad-4ab0-a0b6-5b056333d2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:03.366552 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85638620-76ad-4ab0-a0b6-5b056333d2e2 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:03.367107 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85638620-76ad-4ab0-a0b6-5b056333d2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:03.367595 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85638620-76ad-4ab0-a0b6-5b056333d2e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:03.369791 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:03.376450 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85638620-76ad-4ab0-a0b6-5b056333d2e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:03.378660 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85638620-76ad-4ab0-a0b6-5b056333d2e2 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:03.380895 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a69171-8e53-45b7-af88-a03f8c82d661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:03.381539 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a69171-8e53-45b7-af88-a03f8c82d661 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:03.381871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a69171-8e53-45b7-af88-a03f8c82d661 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:03.382141 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a69171-8e53-45b7-af88-a03f8c82d661 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:03.393274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-85638620-76ad-4ab0-a0b6-5b056333d2e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:03.393274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:03.393274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:03.393274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:03.393274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:03.393274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:03.393274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:03.393274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:03.393274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:03.393274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:03.393274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:03.393274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:03.393274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:03.393274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:03.393274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:03.394212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:03.394212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:03.394212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:03.394212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:03.394212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:03.394212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:03.394212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:03.394212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:03.394212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:03.394212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:03.394212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:03.394212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:03.394212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:03.394212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:03.394212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:03.394212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:03.394212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:03.395111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:03.395111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:03.395111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:03.395111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:03.395111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:03.395111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:03.395111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:03.395111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:03.395563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a69171-8e53-45b7-af88-a03f8c82d661 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:00:03.395641 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 309/608] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:03.400418 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74aaa5-7873-4ee4-934b-e919046df187 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:03.400418 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74aaa5-7873-4ee4-934b-e919046df187 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:03.400418 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74aaa5-7873-4ee4-934b-e919046df187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:03.400741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-04a69171-8e53-45b7-af88-a03f8c82d661 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:00:03.400741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:03.400741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:03.400741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:03.400741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:03.400741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:03.400741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:03.400741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:03.400741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:03.400741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:03.400741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:03.400741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:03.400741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:00:03.400741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:00:03.400741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:00:03.400741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:00:03.401837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:00:03.401837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:00:03.401837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:00:03.401837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:00:03.401837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:03.401837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:03.401837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:03.401837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:03.401837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:03.401837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:03.401837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:03.401837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:03.401837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:03.401837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:03.401837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:03.401837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:03.402970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:03.402970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:03.402970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:03.402970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:03.402970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:00:03.402970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:00:03.402970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:00:03.402970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:00:03.402970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:00:03.402970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:00:03.402970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:00:03.402970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:03.402970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 300/609] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 20:00:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:03.406238 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a69171-8e53-45b7-af88-a03f8c82d661 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:03.420170 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-3e74aaa5-7873-4ee4-934b-e919046df187 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:03.421687 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-3e74aaa5-7873-4ee4-934b-e919046df187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:03.509966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 310/610] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:00:03.511073 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74aaa5-7873-4ee4-934b-e919046df187 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:03.531920 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a69171-8e53-45b7-af88-a03f8c82d661 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:03.599976 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003168ed-5859-4a0f-b919-ddcb30ca3d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:03.600605 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003168ed-5859-4a0f-b919-ddcb30ca3d6c None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:03.600887 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003168ed-5859-4a0f-b919-ddcb30ca3d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:03.601166 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003168ed-5859-4a0f-b919-ddcb30ca3d6c None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:03.601918 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003168ed-5859-4a0f-b919-ddcb30ca3d6c None admin] RBAC: Adding query filter params (name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:03.602223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003168ed-5859-4a0f-b919-ddcb30ca3d6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:03.603593 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003168ed-5859-4a0f-b919-ddcb30ca3d6c None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:03.613056 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:03.619806 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 301/611] 10.222.0.73 () {56 vars in 1103 bytes} [Mon Aug 29 20:00:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:03.625784 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003168ed-5859-4a0f-b919-ddcb30ca3d6c None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:03.645625 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74aaa5-7873-4ee4-934b-e919046df187 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:03.709149 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:03.720447 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad850f-16b4-4c51-9344-c6fb3434cc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:03.721223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad850f-16b4-4c51-9344-c6fb3434cc43 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:03.721510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad850f-16b4-4c51-9344-c6fb3434cc43 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:03.721808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad850f-16b4-4c51-9344-c6fb3434cc43 None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:03.722509 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aad850f-16b4-4c51-9344-c6fb3434cc43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:03.722917 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aad850f-16b4-4c51-9344-c6fb3434cc43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:03.724461 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aad850f-16b4-4c51-9344-c6fb3434cc43 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:03.749970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 311/612] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:00:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:03.755977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad850f-16b4-4c51-9344-c6fb3434cc43 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:03.764418 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003168ed-5859-4a0f-b919-ddcb30ca3d6c None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:03.817321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415a7c5-ce10-4033-90e7-73702f918adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:03.817999 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415a7c5-ce10-4033-90e7-73702f918adf None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:03.818284 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415a7c5-ce10-4033-90e7-73702f918adf None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:03.818559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415a7c5-ce10-4033-90e7-73702f918adf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:03.842414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415a7c5-ce10-4033-90e7-73702f918adf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:03.844986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415a7c5-ce10-4033-90e7-73702f918adf None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:03.872462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-3415a7c5-ce10-4033-90e7-73702f918adf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:03.872462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:03.872462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:03.872462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:03.872462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:03.872462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:03.872462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:03.872462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:03.872462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:03.872462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:03.872462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:03.872462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:03.872462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:03.872462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:03.872462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:03.872462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:03.873788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:03.873788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:03.873788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:03.873788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:03.873788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:03.873788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:03.873788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:03.873788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:03.873788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:03.873788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:03.873788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:03.873788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:03.873788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:03.873788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:03.873788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:03.873788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:03.873788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:03.874676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:03.874676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:03.874676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:03.874676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 302/613] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:03.881557 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415a7c5-ce10-4033-90e7-73702f918adf None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:03.931089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad850f-16b4-4c51-9344-c6fb3434cc43 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:03.968128 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0116b-4fde-482c-a97f-533713a97a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:03.969170 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0116b-4fde-482c-a97f-533713a97a0e None admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:03.969685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0116b-4fde-482c-a97f-533713a97a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:03.970277 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0116b-4fde-482c-a97f-533713a97a0e None admin] PATH_INFO: `/v3/projects/2f5ec615d98548159487c97cead336a0/users/64ba58d774674ed7b233ad6416db068a/roles/8ca2ec70f49a4f4da3ce9f3ac0b73083` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:03.985979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d0116b-4fde-482c-a97f-533713a97a0e None admin] RBAC: Authorizing `identity:create_grant(project_id=2f5ec615d98548159487c97cead336a0, user_id=64ba58d774674ed7b233ad6416db068a, role_id=8ca2ec70f49a4f4da3ce9f3ac0b73083)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:03.987789 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d0116b-4fde-482c-a97f-533713a97a0e None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:03.990400 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:04.009876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 312/614] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 20:00:03 2022] PUT /identity/v3/projects/2f5ec615d98548159487c97cead336a0/users/64ba58d774674ed7b233ad6416db068a/roles/8ca2ec70f49a4f4da3ce9f3ac0b73083 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:00:04.011243 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0116b-4fde-482c-a97f-533713a97a0e None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:04.012744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415a7c5-ce10-4033-90e7-73702f918adf None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:04.015269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:04.057677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102cd20-8c54-40f1-a63f-4b8984513ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:04.058452 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102cd20-8c54-40f1-a63f-4b8984513ceb None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:04.059183 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102cd20-8c54-40f1-a63f-4b8984513ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:04.059540 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102cd20-8c54-40f1-a63f-4b8984513ceb None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:04.061827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8102cd20-8c54-40f1-a63f-4b8984513ceb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:04.061827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8102cd20-8c54-40f1-a63f-4b8984513ceb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:04.062861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8102cd20-8c54-40f1-a63f-4b8984513ceb None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:04.081354 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 303/615] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:04.090273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102cd20-8c54-40f1-a63f-4b8984513ceb None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:04.225577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102cd20-8c54-40f1-a63f-4b8984513ceb None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:04.262554 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0116b-4fde-482c-a97f-533713a97a0e None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:04.276401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9aed5-4b7b-4410-bbd9-995ecb1ffa09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:04.277899 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9aed5-4b7b-4410-bbd9-995ecb1ffa09 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:04.278379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9aed5-4b7b-4410-bbd9-995ecb1ffa09 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:04.278757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9aed5-4b7b-4410-bbd9-995ecb1ffa09 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:04.290039 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9aed5-4b7b-4410-bbd9-995ecb1ffa09 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:04.291633 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9aed5-4b7b-4410-bbd9-995ecb1ffa09 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:04.309201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-75f9aed5-4b7b-4410-bbd9-995ecb1ffa09 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:00:04.309201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:04.309201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:04.309201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:04.309201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:04.309201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:04.309201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:04.309201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:04.309201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:04.309201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:04.309201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:04.309201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:04.309201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:00:04.309201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:00:04.309201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:00:04.309201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:00:04.310880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:04.310880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:04.310880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:04.310880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:04.310880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:04.310880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:04.310880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:04.310880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:04.310880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:04.310880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:04.310880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:04.310880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:04.310880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:04.310880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:04.310880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:04.310880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:04.310880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:04.310880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:04.311954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:04.311954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:04.311954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:00:04.311954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:00:04.311954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:00:04.311954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:00:04.311954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:00:04.311954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:00:04.311954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:00:04.311954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:04.311954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 304/616] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 20:00:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:04.314794 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd4624-fa75-461b-a02a-ff258bb865f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:04.315442 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd4624-fa75-461b-a02a-ff258bb865f6 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:04.315873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd4624-fa75-461b-a02a-ff258bb865f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:04.315989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd4624-fa75-461b-a02a-ff258bb865f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:04.317033 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9aed5-4b7b-4410-bbd9-995ecb1ffa09 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:04.329592 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd4624-fa75-461b-a02a-ff258bb865f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:00:04.335294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-23fd4624-fa75-461b-a02a-ff258bb865f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:00:04.335294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:04.335294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:04.335294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:04.335294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:04.335294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:04.335294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:04.335294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:04.335294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:04.335294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:04.335294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:04.335294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:04.335294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:00:04.335294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:00:04.335294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:00:04.335294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:00:04.336256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:00:04.336256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:00:04.336256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:00:04.336256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:00:04.336256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:04.336256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:04.336256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:04.336256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:04.336256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:04.336256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:04.336256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:04.336256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:04.336256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:04.336256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:04.336256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:04.336256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:04.337143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:04.337143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:04.337143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:04.337143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:04.337143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:00:04.337143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:00:04.337143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:00:04.337143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:00:04.337143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:00:04.337143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:00:04.337143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:00:04.337143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:04.337143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 313/617] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 20:00:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:04.342862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd4624-fa75-461b-a02a-ff258bb865f6 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:04.352611 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:04.415519 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:04.471479 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd4624-fa75-461b-a02a-ff258bb865f6 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:04.472616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9aed5-4b7b-4410-bbd9-995ecb1ffa09 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:04.511155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870cb49-0925-4fec-953b-41eb17b49825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:04.511688 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870cb49-0925-4fec-953b-41eb17b49825 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:04.511970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870cb49-0925-4fec-953b-41eb17b49825 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:04.512238 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870cb49-0925-4fec-953b-41eb17b49825 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:04.512939 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870cb49-0925-4fec-953b-41eb17b49825 None admin] RBAC: Adding query filter params (name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:04.513246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870cb49-0925-4fec-953b-41eb17b49825 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:04.514672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870cb49-0925-4fec-953b-41eb17b49825 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:04.523783 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 314/618] 10.222.0.73 () {56 vars in 1103 bytes} [Mon Aug 29 20:00:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:04.525468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc9401-8d22-47b8-a581-10859065a724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:04.526356 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc9401-8d22-47b8-a581-10859065a724 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:04.526866 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc9401-8d22-47b8-a581-10859065a724 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:04.527371 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc9401-8d22-47b8-a581-10859065a724 None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:04.528858 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc9401-8d22-47b8-a581-10859065a724 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:04.529403 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc9401-8d22-47b8-a581-10859065a724 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:04.529766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870cb49-0925-4fec-953b-41eb17b49825 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:04.531436 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc9401-8d22-47b8-a581-10859065a724 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:04.562961 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 305/619] 10.222.0.73 () {56 vars in 1135 bytes} [Mon Aug 29 20:00:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:04.569164 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc9401-8d22-47b8-a581-10859065a724 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:04.666859 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:04.694213 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc9401-8d22-47b8-a581-10859065a724 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:04.736218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec66692-ce3f-4b01-9f58-edb77f88ecd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:04.737753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec66692-ce3f-4b01-9f58-edb77f88ecd9 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:04.738422 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec66692-ce3f-4b01-9f58-edb77f88ecd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:04.738902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec66692-ce3f-4b01-9f58-edb77f88ecd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:04.749346 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870cb49-0925-4fec-953b-41eb17b49825 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:04.753958 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec66692-ce3f-4b01-9f58-edb77f88ecd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:04.755667 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec66692-ce3f-4b01-9f58-edb77f88ecd9 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:04.771195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:04.782861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-5ec66692-ce3f-4b01-9f58-edb77f88ecd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:04.782861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:04.782861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:04.782861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:04.782861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:04.782861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:04.782861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:04.782861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:04.782861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:04.782861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:04.782861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:04.782861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:04.782861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:04.782861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:04.782861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:04.782861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:04.783881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:04.783881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:04.783881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:04.783881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:04.783881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:04.783881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:04.783881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:04.783881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:04.783881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:04.783881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:04.783881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:04.783881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:04.783881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:04.783881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:04.783881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:04.783881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:04.783881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:04.784797 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:04.784797 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:04.784797 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:04.787823 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 306/620] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:04.793595 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec66692-ce3f-4b01-9f58-edb77f88ecd9 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:04.804140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9efec-2bdc-41c7-9fc5-1226d0a59bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:04.805103 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9efec-2bdc-41c7-9fc5-1226d0a59bab None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:04.805604 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9efec-2bdc-41c7-9fc5-1226d0a59bab None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:04.806148 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9efec-2bdc-41c7-9fc5-1226d0a59bab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:04.820966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9efec-2bdc-41c7-9fc5-1226d0a59bab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:04.822838 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9efec-2bdc-41c7-9fc5-1226d0a59bab None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:04.841273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-89f9efec-2bdc-41c7-9fc5-1226d0a59bab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:04.841273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:04.841273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:04.841273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:04.841273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:04.841273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:04.841273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:04.841273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:04.841273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:04.841273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:04.841273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:04.841273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:04.841273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:04.841273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:04.841273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:04.841273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:04.842330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:04.842330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:04.842330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:04.842330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:04.842330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:04.842330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:04.842330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:04.842330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:04.842330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:04.842330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:04.842330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:04.842330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:04.842330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:04.842330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:04.842330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:04.842330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:04.842330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:04.843245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:04.843245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:04.843245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:04.845215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 315/621] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:00:04.851092 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9efec-2bdc-41c7-9fc5-1226d0a59bab None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:04.964971 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9efec-2bdc-41c7-9fc5-1226d0a59bab None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:05.024242 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d2bfa-e281-4286-827f-f0e01798847a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:05.024777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d2bfa-e281-4286-827f-f0e01798847a None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:05.025053 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d2bfa-e281-4286-827f-f0e01798847a None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:05.025323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d2bfa-e281-4286-827f-f0e01798847a None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:05.025993 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d2bfa-e281-4286-827f-f0e01798847a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:05.026295 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d2bfa-e281-4286-827f-f0e01798847a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:05.027673 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d2bfa-e281-4286-827f-f0e01798847a None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:05.048055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 316/622] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:05.054104 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec66692-ce3f-4b01-9f58-edb77f88ecd9 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:05.056487 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d2bfa-e281-4286-827f-f0e01798847a None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:05.107993 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:05.108634 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:05.118710 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11a2b3-7f4b-4934-8778-28eb83924d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:05.119317 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11a2b3-7f4b-4934-8778-28eb83924d92 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:05.119591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11a2b3-7f4b-4934-8778-28eb83924d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:05.119856 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11a2b3-7f4b-4934-8778-28eb83924d92 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:05.120496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f11a2b3-7f4b-4934-8778-28eb83924d92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:05.120797 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f11a2b3-7f4b-4934-8778-28eb83924d92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:05.122216 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f11a2b3-7f4b-4934-8778-28eb83924d92 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:05.136054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 307/623] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:05.142401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11a2b3-7f4b-4934-8778-28eb83924d92 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:05.231123 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d2bfa-e281-4286-827f-f0e01798847a None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:05.271524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c9b70-dcc3-4a70-9128-f1d5ac069a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:05.272178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c9b70-dcc3-4a70-9128-f1d5ac069a03 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:05.272425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c9b70-dcc3-4a70-9128-f1d5ac069a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:05.272699 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c9b70-dcc3-4a70-9128-f1d5ac069a03 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:05.280945 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c9b70-dcc3-4a70-9128-f1d5ac069a03 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:05.282386 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c9b70-dcc3-4a70-9128-f1d5ac069a03 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:05.288265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11a2b3-7f4b-4934-8778-28eb83924d92 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:05.294609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-372c9b70-dcc3-4a70-9128-f1d5ac069a03 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:00:05.294609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:05.294609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:05.294609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:05.294609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:05.294609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:05.294609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:05.294609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:05.294609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:05.294609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:05.294609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:05.294609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:05.294609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:00:05.294609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:00:05.294609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:00:05.294609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:00:05.295706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:05.295706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:05.295706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:05.295706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:05.295706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:05.295706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:05.295706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:05.295706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:05.295706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:05.295706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:05.295706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:05.295706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:05.295706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:05.295706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:05.295706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:05.295706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:05.295706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:05.295706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:05.296650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:05.296650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:05.296650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:00:05.296650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:00:05.296650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:00:05.296650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:00:05.296650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:00:05.296650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:00:05.296650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:00:05.296650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:05.296650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 317/624] 10.222.0.73 () {56 vars in 1093 bytes} [Mon Aug 29 20:00:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:05.297233 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ec64a-a272-41a4-82eb-b72e29bca8dc None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:05.302111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ec64a-a272-41a4-82eb-b72e29bca8dc None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:05.302111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ec64a-a272-41a4-82eb-b72e29bca8dc None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:05.302111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 318/625] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:05.302723 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ec64a-a272-41a4-82eb-b72e29bca8dc None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:05.336560 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed2719-2175-4bf7-b329-3c03504979f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:05.337234 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed2719-2175-4bf7-b329-3c03504979f1 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:05.337795 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed2719-2175-4bf7-b329-3c03504979f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:05.338072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed2719-2175-4bf7-b329-3c03504979f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:05.348725 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed2719-2175-4bf7-b329-3c03504979f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:05.350264 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed2719-2175-4bf7-b329-3c03504979f1 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:05.364064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-45ed2719-2175-4bf7-b329-3c03504979f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:05.364064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:05.364064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:05.364064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:05.364064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:05.364064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:05.364064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:05.364064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:05.364064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:05.364064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:05.364064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:05.364064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:05.364064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:05.364064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:05.364064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:05.366218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:05.366218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:05.366218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:05.366218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:05.366218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:05.366218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:05.366218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:05.366218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:05.366218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:05.366218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:05.366218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:05.366218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:05.366218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:05.366218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:05.366218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:05.366218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:05.366218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:05.368363 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:05.368363 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:05.368363 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:05.368363 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:05.368363 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:05.368363 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:05.368363 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:05.368363 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:05.368363 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 308/626] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:05.368363 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383c015-fca3-42b4-b9fe-4d46c91840e6 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:05.368363 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383c015-fca3-42b4-b9fe-4d46c91840e6 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:05.369778 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383c015-fca3-42b4-b9fe-4d46c91840e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:05.392945 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-0383c015-fca3-42b4-b9fe-4d46c91840e6 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:05.394248 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-0383c015-fca3-42b4-b9fe-4d46c91840e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:05.417985 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:05.433792 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ec64a-a272-41a4-82eb-b72e29bca8dc None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:05.448359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:05.472580 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 309/627] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:00:05.473281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383c015-fca3-42b4-b9fe-4d46c91840e6 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:05.484058 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446e0b8-fcc2-4cd7-8e54-94df6fd7d7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:05.485108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446e0b8-fcc2-4cd7-8e54-94df6fd7d7d1 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:05.485604 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446e0b8-fcc2-4cd7-8e54-94df6fd7d7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:05.486333 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446e0b8-fcc2-4cd7-8e54-94df6fd7d7d1 None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:05.487471 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446e0b8-fcc2-4cd7-8e54-94df6fd7d7d1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:05.488015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446e0b8-fcc2-4cd7-8e54-94df6fd7d7d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:05.490007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446e0b8-fcc2-4cd7-8e54-94df6fd7d7d1 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:05.512775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 319/628] 10.222.0.73 () {56 vars in 1133 bytes} [Mon Aug 29 20:00:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:05.519116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446e0b8-fcc2-4cd7-8e54-94df6fd7d7d1 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:05.586519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383c015-fca3-42b4-b9fe-4d46c91840e6 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:05.624226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704de872-4583-4bbd-bcad-f971328bea29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:05.624695 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704de872-4583-4bbd-bcad-f971328bea29 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:05.624977 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704de872-4583-4bbd-bcad-f971328bea29 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:05.625256 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704de872-4583-4bbd-bcad-f971328bea29 None admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:05.625996 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704de872-4583-4bbd-bcad-f971328bea29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:05.626270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704de872-4583-4bbd-bcad-f971328bea29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:05.627800 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704de872-4583-4bbd-bcad-f971328bea29 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:05.634939 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446e0b8-fcc2-4cd7-8e54-94df6fd7d7d1 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:05.640635 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 310/629] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:00:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:05.646673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617c686-a57f-46ac-8122-1e6faaa3c219 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:05.646982 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617c686-a57f-46ac-8122-1e6faaa3c219 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:05.647265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617c686-a57f-46ac-8122-1e6faaa3c219 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:05.648922 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 311/630] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:05.649646 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617c686-a57f-46ac-8122-1e6faaa3c219 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:05.675983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5449fd-6ebc-41ec-b0f2-723a61a3afbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:05.677080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5449fd-6ebc-41ec-b0f2-723a61a3afbf None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:05.677663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5449fd-6ebc-41ec-b0f2-723a61a3afbf None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:05.678175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5449fd-6ebc-41ec-b0f2-723a61a3afbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:05.698405 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:05.699666 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:05.761858 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5449fd-6ebc-41ec-b0f2-723a61a3afbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:05.763870 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5449fd-6ebc-41ec-b0f2-723a61a3afbf None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:05.764407 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617c686-a57f-46ac-8122-1e6faaa3c219 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:05.801458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00d5ae-d111-4b57-bd22-e0bd3bad1739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:05.802526 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00d5ae-d111-4b57-bd22-e0bd3bad1739 None admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:05.803041 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00d5ae-d111-4b57-bd22-e0bd3bad1739 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:05.803532 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00d5ae-d111-4b57-bd22-e0bd3bad1739 None admin] PATH_INFO: `/v3/projects/2f5ec615d98548159487c97cead336a0/users/b92e30da8f394f3abc19b9e86ef126e5/roles/8ca2ec70f49a4f4da3ce9f3ac0b73083` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:05.818782 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be00d5ae-d111-4b57-bd22-e0bd3bad1739 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f5ec615d98548159487c97cead336a0, user_id=b92e30da8f394f3abc19b9e86ef126e5, role_id=8ca2ec70f49a4f4da3ce9f3ac0b73083)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:05.820226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be00d5ae-d111-4b57-bd22-e0bd3bad1739 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:05.837856 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 312/631] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 20:00:05 2022] PUT /identity/v3/projects/2f5ec615d98548159487c97cead336a0/users/b92e30da8f394f3abc19b9e86ef126e5/roles/8ca2ec70f49a4f4da3ce9f3ac0b73083 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:00:05.837856 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cea6e6-084a-407f-8101-e880727e6d97 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:05.837856 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cea6e6-084a-407f-8101-e880727e6d97 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:05.837856 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cea6e6-084a-407f-8101-e880727e6d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:05.856894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-27cea6e6-084a-407f-8101-e880727e6d97 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:05.858676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-27cea6e6-084a-407f-8101-e880727e6d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:05.894095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 320/632] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:00:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 20:00:05.895476 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5449fd-6ebc-41ec-b0f2-723a61a3afbf None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:05.944779 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 313/633] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:00:05.946467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cea6e6-084a-407f-8101-e880727e6d97 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:05.955067 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:05.983119 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:06.029040 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5449fd-6ebc-41ec-b0f2-723a61a3afbf None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:06.071226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934597c-277d-490e-8c27-a4ebe916636c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:06.072253 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934597c-277d-490e-8c27-a4ebe916636c None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:06.072735 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934597c-277d-490e-8c27-a4ebe916636c None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:06.073204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934597c-277d-490e-8c27-a4ebe916636c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:06.079030 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cea6e6-084a-407f-8101-e880727e6d97 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:06.089357 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934597c-277d-490e-8c27-a4ebe916636c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:00:06.098644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-4934597c-277d-490e-8c27-a4ebe916636c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:00:06.098644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:06.098644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:06.098644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:06.098644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:06.098644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:06.098644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:06.098644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:06.098644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:06.098644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:06.098644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:06.098644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:06.098644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:00:06.098644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:00:06.098644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:00:06.098644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:00:06.099639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:00:06.099639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:00:06.099639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:00:06.099639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:00:06.099639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:06.099639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:06.099639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:06.099639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:06.099639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:06.099639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:06.099639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:06.099639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:06.099639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:06.099639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:06.099639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:06.099639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:06.100466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:06.100466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:06.100466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:06.100466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:06.100466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:00:06.100466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:00:06.100466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:00:06.100466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:00:06.100466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:00:06.100466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:00:06.100466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:00:06.100466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:06.103022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 321/634] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 20:00:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:06.110505 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934597c-277d-490e-8c27-a4ebe916636c None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:06.125169 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e34bf-36e8-4e80-b677-aa5ecb2a09f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:06.125880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e34bf-36e8-4e80-b677-aa5ecb2a09f9 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:06.126164 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e34bf-36e8-4e80-b677-aa5ecb2a09f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:06.126440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e34bf-36e8-4e80-b677-aa5ecb2a09f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:06.138502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e34bf-36e8-4e80-b677-aa5ecb2a09f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:06.139979 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e34bf-36e8-4e80-b677-aa5ecb2a09f9 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:06.156536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-bd7e34bf-36e8-4e80-b677-aa5ecb2a09f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:06.156536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:06.156536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:06.156536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:06.156536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:06.156536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:06.156536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:06.156536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:06.156536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:06.156536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:06.156536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:06.156536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:06.156536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:06.156536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:06.156536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:06.156536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:06.157519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:06.157519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:06.157519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:06.157519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:06.157519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:06.157519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:06.157519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:06.157519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:06.157519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:06.157519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:06.157519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:06.157519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:06.157519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:06.157519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:06.157519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:06.157519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:06.157519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:06.158454 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:06.158454 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:06.158454 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:06.158454 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 314/635] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:06.163382 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e34bf-36e8-4e80-b677-aa5ecb2a09f9 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:06.242672 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:06.252573 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934597c-277d-490e-8c27-a4ebe916636c None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:06.267826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:06.269949 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e34bf-36e8-4e80-b677-aa5ecb2a09f9 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:06.305921 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd8e68-7684-4abe-82f0-7cdc18a469b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:06.306543 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd8e68-7684-4abe-82f0-7cdc18a469b0 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:06.306810 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd8e68-7684-4abe-82f0-7cdc18a469b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:06.307121 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd8e68-7684-4abe-82f0-7cdc18a469b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:06.307950 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd8e68-7684-4abe-82f0-7cdc18a469b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:06.308259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd8e68-7684-4abe-82f0-7cdc18a469b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:06.309673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd8e68-7684-4abe-82f0-7cdc18a469b0 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:06.322580 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 315/636] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:06.328508 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd8e68-7684-4abe-82f0-7cdc18a469b0 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:06.334110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ad6e3-b767-460e-91f6-9476fda8bb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:06.335294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ad6e3-b767-460e-91f6-9476fda8bb4c None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:06.335725 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ad6e3-b767-460e-91f6-9476fda8bb4c None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:06.336140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ad6e3-b767-460e-91f6-9476fda8bb4c None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:06.336963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ad6e3-b767-460e-91f6-9476fda8bb4c None admin] RBAC: Adding query filter params (name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:06.337411 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ad6e3-b767-460e-91f6-9476fda8bb4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:06.339029 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ad6e3-b767-460e-91f6-9476fda8bb4c None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:06.350304 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 322/637] 10.222.0.73 () {56 vars in 1103 bytes} [Mon Aug 29 20:00:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:06.351462 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ad6e3-b767-460e-91f6-9476fda8bb4c None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:06.447250 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd8e68-7684-4abe-82f0-7cdc18a469b0 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:06.462040 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ad6e3-b767-460e-91f6-9476fda8bb4c None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:06.494411 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc9809-bd7a-4569-8152-70fa9f3890ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:06.495044 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc9809-bd7a-4569-8152-70fa9f3890ea None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:06.495315 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc9809-bd7a-4569-8152-70fa9f3890ea None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:06.495633 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc9809-bd7a-4569-8152-70fa9f3890ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:06.497929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77fc24-3bb8-4e20-b15f-be13584d6acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:06.498762 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:06.498871 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77fc24-3bb8-4e20-b15f-be13584d6acd None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:06.499094 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77fc24-3bb8-4e20-b15f-be13584d6acd None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:06.499363 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77fc24-3bb8-4e20-b15f-be13584d6acd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:06.504530 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc9809-bd7a-4569-8152-70fa9f3890ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:06.506713 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc9809-bd7a-4569-8152-70fa9f3890ea None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:06.510877 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77fc24-3bb8-4e20-b15f-be13584d6acd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:00:06.516240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-9e77fc24-3bb8-4e20-b15f-be13584d6acd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:00:06.516240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:06.516240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:06.516240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:06.516240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:06.516240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:06.516240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:06.516240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:06.516240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:06.516240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:06.516240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:06.516240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:06.516240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:00:06.516240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:00:06.516240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:00:06.516240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:00:06.517210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:00:06.517210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:00:06.517210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:00:06.517210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:00:06.517210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:06.517210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:06.517210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:06.517210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:06.517210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:06.517210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:06.517210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:06.517210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:06.517210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:06.517210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:06.517210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:06.517210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:06.523157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:06.523157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:06.523157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:06.523157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:06.523157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:00:06.523157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:00:06.523157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:00:06.523157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:00:06.523157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:00:06.523157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:00:06.523157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:00:06.523157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:06.523157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 323/638] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 20:00:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:06.523157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77fc24-3bb8-4e20-b15f-be13584d6acd None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:06.523908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-c5bc9809-bd7a-4569-8152-70fa9f3890ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:06.523908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:06.523908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:06.523908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:06.523908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:06.523908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:06.523908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:06.523908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:06.523908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:06.523908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:06.523908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:06.523908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:06.523908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:06.523908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:06.523908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:06.524716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:06.524716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:06.524716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:06.524716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:06.524716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:06.524716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:06.524716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:06.524716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:06.524716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:06.524716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:06.524716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:06.524716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:06.524716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:06.524716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:06.524716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:06.524716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:06.524716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:06.525577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:06.525577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:06.525577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:06.525577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:06.525577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:06.525577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:06.525577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:06.525577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:06.525577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 316/639] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:06.525577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809c627-8594-4e68-b8b4-130a108213c2 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:06.525577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809c627-8594-4e68-b8b4-130a108213c2 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:06.527615 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809c627-8594-4e68-b8b4-130a108213c2 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:06.528509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 317/640] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:06.529208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809c627-8594-4e68-b8b4-130a108213c2 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:06.564472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:06.644844 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77fc24-3bb8-4e20-b15f-be13584d6acd None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:06.645843 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809c627-8594-4e68-b8b4-130a108213c2 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:06.681473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01318afa-419e-43cd-82f0-80c0129838c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:06.682141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01318afa-419e-43cd-82f0-80c0129838c5 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:06.682450 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01318afa-419e-43cd-82f0-80c0129838c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:06.682691 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01318afa-419e-43cd-82f0-80c0129838c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:06.684050 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a7ef7-22df-4c0f-ad4a-fdbd9d054113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:06.684691 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a7ef7-22df-4c0f-ad4a-fdbd9d054113 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:06.685483 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a7ef7-22df-4c0f-ad4a-fdbd9d054113 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:06.685783 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a7ef7-22df-4c0f-ad4a-fdbd9d054113 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:06.686447 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664a7ef7-22df-4c0f-ad4a-fdbd9d054113 None admin] RBAC: Adding query filter params (name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:06.686751 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664a7ef7-22df-4c0f-ad4a-fdbd9d054113 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:06.688166 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664a7ef7-22df-4c0f-ad4a-fdbd9d054113 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:06.695784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01318afa-419e-43cd-82f0-80c0129838c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:06.697234 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01318afa-419e-43cd-82f0-80c0129838c5 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:06.703151 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 318/641] 10.222.0.73 () {56 vars in 1103 bytes} [Mon Aug 29 20:00:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:06.703151 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a7ef7-22df-4c0f-ad4a-fdbd9d054113 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:06.718151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-01318afa-419e-43cd-82f0-80c0129838c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:06.718151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:06.718151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:06.718151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:06.718151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:06.718151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:06.718151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:06.718151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:06.718151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:06.718151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:06.718151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:06.718151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:06.718151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:06.718151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:06.718151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:06.718151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:06.720252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:06.720252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:06.720252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:06.720252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:06.720252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:06.720252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:06.720252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:06.720252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:06.720252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:06.720252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:06.720252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:06.720252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:06.720252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:06.720252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:06.720252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:06.720252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:06.720252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:06.721174 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:06.721174 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:06.721174 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:06.721174 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 324/642] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:06.721174 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b13fd-4f60-4ca9-a8e5-5bc78db605d4 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:06.721174 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b13fd-4f60-4ca9-a8e5-5bc78db605d4 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:06.721174 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b13fd-4f60-4ca9-a8e5-5bc78db605d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:06.740018 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-748b13fd-4f60-4ca9-a8e5-5bc78db605d4 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:06.741227 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-748b13fd-4f60-4ca9-a8e5-5bc78db605d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:06.770346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:06.819107 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a7ef7-22df-4c0f-ad4a-fdbd9d054113 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:06.829904 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 325/643] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:00:06.829904 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b13fd-4f60-4ca9-a8e5-5bc78db605d4 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:06.844249 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:06.860616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017609c2-e19d-404d-994a-6590803ad5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:06.861230 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017609c2-e19d-404d-994a-6590803ad5b4 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:06.861557 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017609c2-e19d-404d-994a-6590803ad5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:06.861876 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017609c2-e19d-404d-994a-6590803ad5b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:06.862621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017609c2-e19d-404d-994a-6590803ad5b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:06.862924 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017609c2-e19d-404d-994a-6590803ad5b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:06.864458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017609c2-e19d-404d-994a-6590803ad5b4 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:06.877646 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 319/644] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:00:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:06.878664 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017609c2-e19d-404d-994a-6590803ad5b4 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:06.961494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b13fd-4f60-4ca9-a8e5-5bc78db605d4 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:07.003887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2177637-c758-4c6a-a1d0-164c48507c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:07.005455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2177637-c758-4c6a-a1d0-164c48507c46 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:07.005991 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2177637-c758-4c6a-a1d0-164c48507c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:07.006479 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2177637-c758-4c6a-a1d0-164c48507c46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:07.007359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017609c2-e19d-404d-994a-6590803ad5b4 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:07.020671 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2177637-c758-4c6a-a1d0-164c48507c46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:07.022691 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2177637-c758-4c6a-a1d0-164c48507c46 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:07.026775 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:07.044925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-b2177637-c758-4c6a-a1d0-164c48507c46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:07.044925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:07.044925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:07.044925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:07.044925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:07.044925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:07.044925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:07.044925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:07.044925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:07.044925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:07.044925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:07.044925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:07.044925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:07.044925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:07.044925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:07.044925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:07.046034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:07.046034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:07.046034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:07.046034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:07.046034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:07.046034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:07.046034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:07.046034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:07.046034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:07.046034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:07.046034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:07.046034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:07.046034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:07.046034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:07.046034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:07.046034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:07.046034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:07.047072 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:07.047072 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:07.047072 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:07.049030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 326/645] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:07.050022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2177637-c758-4c6a-a1d0-164c48507c46 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:07.056019 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1602a-3660-4416-a084-571670b40436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:07.056756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1602a-3660-4416-a084-571670b40436 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:07.057159 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1602a-3660-4416-a084-571670b40436 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:07.057573 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1602a-3660-4416-a084-571670b40436 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:07.058348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1602a-3660-4416-a084-571670b40436 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:07.058777 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1602a-3660-4416-a084-571670b40436 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:07.060601 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1602a-3660-4416-a084-571670b40436 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:07.078405 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 320/646] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:07.078405 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1602a-3660-4416-a084-571670b40436 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:07.142703 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:07.189110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2177637-c758-4c6a-a1d0-164c48507c46 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:07.192230 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1602a-3660-4416-a084-571670b40436 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:07.229827 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697174a9-18cf-4a82-a5e6-fafafcc89aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:07.230356 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697174a9-18cf-4a82-a5e6-fafafcc89aac None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:07.230624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697174a9-18cf-4a82-a5e6-fafafcc89aac None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:07.230899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697174a9-18cf-4a82-a5e6-fafafcc89aac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:07.231667 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697174a9-18cf-4a82-a5e6-fafafcc89aac None admin] RBAC: Adding query filter params (role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=95cf77a92a174ff8b0503baf5c7e63be) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:07.231977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697174a9-18cf-4a82-a5e6-fafafcc89aac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:07.233431 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697174a9-18cf-4a82-a5e6-fafafcc89aac None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:07.237343 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4499ead-a4e8-44b0-8f4b-bbac103fc720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:07.238503 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4499ead-a4e8-44b0-8f4b-bbac103fc720 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:07.239003 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4499ead-a4e8-44b0-8f4b-bbac103fc720 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:07.239538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4499ead-a4e8-44b0-8f4b-bbac103fc720 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:07.239665 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 327/647] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:00:07 2022] GET /identity/v3/role_assignments?user.id=95cf77a92a174ff8b0503baf5c7e63be&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:07.240436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697174a9-18cf-4a82-a5e6-fafafcc89aac None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:07.240723 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4499ead-a4e8-44b0-8f4b-bbac103fc720 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:07.241295 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4499ead-a4e8-44b0-8f4b-bbac103fc720 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:07.242967 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4499ead-a4e8-44b0-8f4b-bbac103fc720 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:07.271656 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 321/648] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:07.272867 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4499ead-a4e8-44b0-8f4b-bbac103fc720 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:07.322159 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:07.392013 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697174a9-18cf-4a82-a5e6-fafafcc89aac None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:07.414506 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4499ead-a4e8-44b0-8f4b-bbac103fc720 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:07.422120 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:07.438834 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3850ab6d-651d-4eb1-8e8f-9ebffbc943fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:07.439478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850ab6d-651d-4eb1-8e8f-9ebffbc943fa None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:07.439759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850ab6d-651d-4eb1-8e8f-9ebffbc943fa None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:07.440031 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850ab6d-651d-4eb1-8e8f-9ebffbc943fa None admin] PATH_INFO: `/v3/users/glance` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:07.450950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3850ab6d-651d-4eb1-8e8f-9ebffbc943fa None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:07.452566 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3850ab6d-651d-4eb1-8e8f-9ebffbc943fa None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:07.465458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-3850ab6d-651d-4eb1-8e8f-9ebffbc943fa None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 20:00:07.465458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:07.465458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:07.465458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:07.465458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:07.465458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:07.465458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:07.465458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:07.465458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:07.465458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:07.465458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:07.465458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:07.465458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:00:07.465458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:00:07.465458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:00:07.465458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:00:07.467078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:07.467078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:07.467078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:07.467078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:07.467078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:07.467078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:07.467078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:07.467078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:07.467078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:07.467078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:07.467078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:07.467078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:07.467078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:07.467078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:07.467078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:07.467078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:07.467078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:07.467078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:07.468475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:07.468475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:07.468475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:00:07.468475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:00:07.468475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:00:07.468475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:00:07.468475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:00:07.468475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:00:07.468475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 20:00:07.468475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:07.468475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 328/649] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 20:00:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:07.469303 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3850ab6d-651d-4eb1-8e8f-9ebffbc943fa None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:07.470578 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2fd48-3897-40ac-8627-f8138d6ab358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:07.471225 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2fd48-3897-40ac-8627-f8138d6ab358 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:07.471506 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2fd48-3897-40ac-8627-f8138d6ab358 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:07.471777 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2fd48-3897-40ac-8627-f8138d6ab358 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:07.490278 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a2fd48-3897-40ac-8627-f8138d6ab358 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:00:07.496013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-36a2fd48-3897-40ac-8627-f8138d6ab358 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:00:07.496013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:07.496013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:07.496013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:07.496013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:07.496013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:07.496013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:07.496013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:07.496013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:07.496013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:07.496013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:07.496013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:07.496013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:00:07.496013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:00:07.496013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:00:07.496013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:00:07.496957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:00:07.496957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:00:07.496957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:00:07.496957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:00:07.496957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:07.496957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:07.496957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:07.496957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:07.496957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:07.496957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:07.496957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:07.496957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:07.496957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:07.496957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:07.496957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:07.496957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:07.499253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:07.499253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:07.499253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:07.499253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:07.499253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:00:07.499253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:00:07.499253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:00:07.499253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:00:07.499253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:00:07.499253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:00:07.499253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:00:07.499253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:07.499253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 322/650] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 20:00:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:07.499253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2fd48-3897-40ac-8627-f8138d6ab358 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:07.601033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3850ab6d-651d-4eb1-8e8f-9ebffbc943fa None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:07.621353 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:07.631942 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2fd48-3897-40ac-8627-f8138d6ab358 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:07.640808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447179f-a00e-4eef-a578-b15f0d2aec40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:07.641557 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447179f-a00e-4eef-a578-b15f0d2aec40 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:07.641951 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447179f-a00e-4eef-a578-b15f0d2aec40 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:07.642299 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447179f-a00e-4eef-a578-b15f0d2aec40 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:07.652192 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8447179f-a00e-4eef-a578-b15f0d2aec40 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:07.654354 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8447179f-a00e-4eef-a578-b15f0d2aec40 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:07.666758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-8447179f-a00e-4eef-a578-b15f0d2aec40 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 20:00:07.666758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:07.666758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:07.666758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:07.666758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:07.666758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:07.666758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:07.666758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:07.666758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:07.666758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:07.666758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:07.666758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:07.666758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:00:07.666758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:00:07.666758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:00:07.666758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:00:07.667780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:07.667780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:07.667780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:07.667780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:07.667780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:07.667780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:07.667780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:07.667780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:07.667780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:07.667780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:07.667780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:07.667780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:07.667780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:07.667780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:07.667780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:07.667780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:07.667780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:07.667780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:07.669215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:07.669215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:07.669215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:00:07.669215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:00:07.669215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:00:07.669215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:00:07.669215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:00:07.669215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:00:07.669215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 20:00:07.669215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:07.669215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 329/651] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 20:00:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:07.669215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447179f-a00e-4eef-a578-b15f0d2aec40 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:07.678396 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afb7d7-d3df-4ed7-8ab1-6e38618bf2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:07.679018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afb7d7-d3df-4ed7-8ab1-6e38618bf2ab None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:07.679302 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afb7d7-d3df-4ed7-8ab1-6e38618bf2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:07.679626 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afb7d7-d3df-4ed7-8ab1-6e38618bf2ab None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:07.680379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afb7d7-d3df-4ed7-8ab1-6e38618bf2ab None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:07.680690 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afb7d7-d3df-4ed7-8ab1-6e38618bf2ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:07.682177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afb7d7-d3df-4ed7-8ab1-6e38618bf2ab None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:07.712059 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 323/652] 10.222.0.73 () {56 vars in 1137 bytes} [Mon Aug 29 20:00:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:07.712059 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afb7d7-d3df-4ed7-8ab1-6e38618bf2ab None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:07.735137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:07.796999 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447179f-a00e-4eef-a578-b15f0d2aec40 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:07.825542 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afb7d7-d3df-4ed7-8ab1-6e38618bf2ab None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:07.833890 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf8101-65fc-4dbb-b3b8-a06c480b7727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:07.834583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf8101-65fc-4dbb-b3b8-a06c480b7727 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:07.834870 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf8101-65fc-4dbb-b3b8-a06c480b7727 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:07.835297 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf8101-65fc-4dbb-b3b8-a06c480b7727 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:07.835983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebf8101-65fc-4dbb-b3b8-a06c480b7727 None admin] RBAC: Adding query filter params (name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:07.836294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebf8101-65fc-4dbb-b3b8-a06c480b7727 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:07.837807 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebf8101-65fc-4dbb-b3b8-a06c480b7727 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:07.848017 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 330/653] 10.222.0.73 () {56 vars in 1103 bytes} [Mon Aug 29 20:00:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:07.848798 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf8101-65fc-4dbb-b3b8-a06c480b7727 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:07.869798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f025d-dd8a-49b0-abfa-ea2493b3130e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:07.871047 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f025d-dd8a-49b0-abfa-ea2493b3130e None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:07.871543 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f025d-dd8a-49b0-abfa-ea2493b3130e None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:07.872043 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f025d-dd8a-49b0-abfa-ea2493b3130e None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:07.873026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f025d-dd8a-49b0-abfa-ea2493b3130e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:07.874130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f025d-dd8a-49b0-abfa-ea2493b3130e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:07.875626 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f025d-dd8a-49b0-abfa-ea2493b3130e None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:07.899381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 324/654] 10.222.0.73 () {56 vars in 1137 bytes} [Mon Aug 29 20:00:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:07.901153 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f025d-dd8a-49b0-abfa-ea2493b3130e None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:07.924645 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:07.980078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf8101-65fc-4dbb-b3b8-a06c480b7727 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:07.990982 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:08.021518 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6427b-39ad-4f00-a44d-c858b494b3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:08.022581 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6427b-39ad-4f00-a44d-c858b494b3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:08.023068 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6427b-39ad-4f00-a44d-c858b494b3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:08.023539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6427b-39ad-4f00-a44d-c858b494b3c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:08.025540 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f025d-dd8a-49b0-abfa-ea2493b3130e None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:08.065011 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294265bd-10cd-4012-9fc9-2186a1bf00de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:08.066361 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294265bd-10cd-4012-9fc9-2186a1bf00de None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:08.066361 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294265bd-10cd-4012-9fc9-2186a1bf00de None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:08.066573 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294265bd-10cd-4012-9fc9-2186a1bf00de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:08.079229 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294265bd-10cd-4012-9fc9-2186a1bf00de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:08.080610 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294265bd-10cd-4012-9fc9-2186a1bf00de None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:08.097985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-294265bd-10cd-4012-9fc9-2186a1bf00de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:08.097985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:08.097985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:08.097985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:08.097985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:08.097985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:08.097985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:08.097985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:08.097985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:08.097985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:08.097985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:08.097985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:08.097985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:08.097985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:08.097985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:08.097985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:08.099018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:08.099018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:08.099018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:08.099018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:08.099018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:08.099018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:08.099018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:08.099018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:08.099018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:08.099018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:08.099018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:08.099018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:08.099018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:08.099018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:08.099018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:08.099018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:08.099018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:08.099994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:08.099994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:08.099994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:08.103928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 325/655] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:08.104811 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294265bd-10cd-4012-9fc9-2186a1bf00de None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:08.106650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6427b-39ad-4f00-a44d-c858b494b3c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:08.109231 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6427b-39ad-4f00-a44d-c858b494b3c8 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:08.171512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:08.218751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 331/656] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:00:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:00:08.218751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3dda9-b79e-418a-84b2-f474683dc041 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:08.218751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3dda9-b79e-418a-84b2-f474683dc041 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:08.218751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3dda9-b79e-418a-84b2-f474683dc041 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:08.218751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 332/657] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:08.218751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3dda9-b79e-418a-84b2-f474683dc041 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:08.223792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294265bd-10cd-4012-9fc9-2186a1bf00de None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:08.229686 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:08.260454 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9674ad3-021f-49ee-aeb1-19de0cec8177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:08.261060 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9674ad3-021f-49ee-aeb1-19de0cec8177 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:08.261349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9674ad3-021f-49ee-aeb1-19de0cec8177 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:08.261672 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9674ad3-021f-49ee-aeb1-19de0cec8177 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:08.331134 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3dda9-b79e-418a-84b2-f474683dc041 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:08.333663 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9674ad3-021f-49ee-aeb1-19de0cec8177 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:08.335173 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9674ad3-021f-49ee-aeb1-19de0cec8177 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:08.366660 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef72d5-8797-45ae-b665-6a62b6412aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:08.367302 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef72d5-8797-45ae-b665-6a62b6412aca None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:08.367544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef72d5-8797-45ae-b665-6a62b6412aca None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:08.367817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef72d5-8797-45ae-b665-6a62b6412aca None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:08.368464 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fef72d5-8797-45ae-b665-6a62b6412aca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:08.368765 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fef72d5-8797-45ae-b665-6a62b6412aca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:08.370191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fef72d5-8797-45ae-b665-6a62b6412aca None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:08.389738 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 333/658] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:08.389738 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef72d5-8797-45ae-b665-6a62b6412aca None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:08.415343 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:08.462774 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:08.464329 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 326/659] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:00:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:00:08.466571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66accc8-35df-4dcf-9347-d80296d11864 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:08.467036 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66accc8-35df-4dcf-9347-d80296d11864 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:08.467349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66accc8-35df-4dcf-9347-d80296d11864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:08.484906 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-d66accc8-35df-4dcf-9347-d80296d11864 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:08.486136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d66accc8-35df-4dcf-9347-d80296d11864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:08.503397 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef72d5-8797-45ae-b665-6a62b6412aca None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:08.545874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25247fcc-1e0f-434b-96a7-4fcc30bf66db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:08.546456 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25247fcc-1e0f-434b-96a7-4fcc30bf66db None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:08.546739 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25247fcc-1e0f-434b-96a7-4fcc30bf66db None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:08.547010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25247fcc-1e0f-434b-96a7-4fcc30bf66db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:08.558031 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25247fcc-1e0f-434b-96a7-4fcc30bf66db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:08.565765 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 327/660] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:00:08.565934 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25247fcc-1e0f-434b-96a7-4fcc30bf66db None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:08.566045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66accc8-35df-4dcf-9347-d80296d11864 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:08.592205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-25247fcc-1e0f-434b-96a7-4fcc30bf66db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:08.592205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:08.592205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:08.592205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:08.592205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:08.592205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:08.592205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:08.592205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:08.592205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:08.592205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:08.592205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:08.592205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:08.592205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:08.592205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:08.592205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:08.592205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:08.593392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:08.593392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:08.593392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:08.593392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:08.593392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:08.593392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:08.593392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:08.593392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:08.593392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:08.593392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:08.593392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:08.593392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:08.593392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:08.593392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:08.593392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:08.593392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:08.593392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:08.594420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:08.594420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:08.594420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:08.595627 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 334/661] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:08.595750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25247fcc-1e0f-434b-96a7-4fcc30bf66db None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:08.676634 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:08.684866 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66accc8-35df-4dcf-9347-d80296d11864 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:08.710266 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25247fcc-1e0f-434b-96a7-4fcc30bf66db None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:08.724218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc55719-c0bf-467d-95b8-66d38df1f4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:08.724860 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc55719-c0bf-467d-95b8-66d38df1f4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:08.725194 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc55719-c0bf-467d-95b8-66d38df1f4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:08.725472 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc55719-c0bf-467d-95b8-66d38df1f4b2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:08.733308 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:08.734314 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc55719-c0bf-467d-95b8-66d38df1f4b2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:08.736232 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc55719-c0bf-467d-95b8-66d38df1f4b2 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:08.750002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-afc55719-c0bf-467d-95b8-66d38df1f4b2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 20:00:08.750002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:08.750002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:08.750002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:08.750002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:08.750002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:08.750002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:08.750002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:08.750002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:08.750002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:08.750002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:08.750002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:08.750002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:00:08.750002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:00:08.750002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:00:08.750002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:00:08.751226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:08.751226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:08.751226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:08.751226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:08.751226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:08.751226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:08.751226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:08.751226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:08.751226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:08.751226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:08.751226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:08.751226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:08.751226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:08.751226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:08.751226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:08.751226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:08.751226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:08.751226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:08.752385 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:08.752385 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:08.752385 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:00:08.752385 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:00:08.752385 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:00:08.752385 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:00:08.752385 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:00:08.752385 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:00:08.752385 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 20:00:08.752385 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:08.754217 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0a9ff-bd53-412c-8154-d72348516ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:08.755050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0a9ff-bd53-412c-8154-d72348516ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:08.755486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0a9ff-bd53-412c-8154-d72348516ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:08.755905 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0a9ff-bd53-412c-8154-d72348516ce6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:08.756034 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 328/662] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 20:00:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:08.756968 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc55719-c0bf-467d-95b8-66d38df1f4b2 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:08.768918 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0a9ff-bd53-412c-8154-d72348516ce6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:08.770678 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0a9ff-bd53-412c-8154-d72348516ce6 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:08.786534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-99c0a9ff-bd53-412c-8154-d72348516ce6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:08.786534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:08.786534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:08.786534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:08.786534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:08.786534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:08.786534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:08.786534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:08.786534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:08.786534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:08.786534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:08.786534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:08.786534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:08.786534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:08.786534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:08.786534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:08.787842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:08.787842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:08.787842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:08.787842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:08.787842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:08.787842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:08.787842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:08.787842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:08.787842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:08.787842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:08.787842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:08.787842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:08.787842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:08.787842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:08.787842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:08.787842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:08.787842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:08.788951 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:08.788951 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:08.788951 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:08.790358 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 335/663] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:08.791314 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0a9ff-bd53-412c-8154-d72348516ce6 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:08.876666 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc55719-c0bf-467d-95b8-66d38df1f4b2 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:08.906212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0a9ff-bd53-412c-8154-d72348516ce6 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:08.925010 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec413d0d-f229-4d89-b0bd-b98d09d83ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:08.925633 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec413d0d-f229-4d89-b0bd-b98d09d83ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:08.926009 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec413d0d-f229-4d89-b0bd-b98d09d83ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:08.926295 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec413d0d-f229-4d89-b0bd-b98d09d83ab6 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:08.926861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec413d0d-f229-4d89-b0bd-b98d09d83ab6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:08.927266 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec413d0d-f229-4d89-b0bd-b98d09d83ab6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:08.928633 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec413d0d-f229-4d89-b0bd-b98d09d83ab6 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:08.945668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 329/664] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:08.945892 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07918f66-2f9f-41dc-8d66-89436d3a3c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:08.946383 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07918f66-2f9f-41dc-8d66-89436d3a3c3a None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:08.946671 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07918f66-2f9f-41dc-8d66-89436d3a3c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:08.946941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07918f66-2f9f-41dc-8d66-89436d3a3c3a None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:08.948016 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07918f66-2f9f-41dc-8d66-89436d3a3c3a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:08.948381 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07918f66-2f9f-41dc-8d66-89436d3a3c3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:08.949850 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07918f66-2f9f-41dc-8d66-89436d3a3c3a None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:08.952159 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec413d0d-f229-4d89-b0bd-b98d09d83ab6 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:08.969371 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:08.969913 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 336/665] 10.222.0.73 () {56 vars in 1139 bytes} [Mon Aug 29 20:00:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:08.970752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07918f66-2f9f-41dc-8d66-89436d3a3c3a None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:09.009284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:09.077289 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec413d0d-f229-4d89-b0bd-b98d09d83ab6 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:09.096946 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07918f66-2f9f-41dc-8d66-89436d3a3c3a None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:09.116947 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480341d3-7a0a-4a2b-81d1-f2081197288b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:09.117607 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480341d3-7a0a-4a2b-81d1-f2081197288b None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:09.117916 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480341d3-7a0a-4a2b-81d1-f2081197288b None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:09.118188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480341d3-7a0a-4a2b-81d1-f2081197288b None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:09.118825 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480341d3-7a0a-4a2b-81d1-f2081197288b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:09.119124 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480341d3-7a0a-4a2b-81d1-f2081197288b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:09.120505 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480341d3-7a0a-4a2b-81d1-f2081197288b None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:09.134679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 330/666] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:09.135259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480341d3-7a0a-4a2b-81d1-f2081197288b None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:09.139867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ca3b8-1754-4bd8-9f7c-c95c948f5c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:09.141293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ca3b8-1754-4bd8-9f7c-c95c948f5c1f None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:09.141293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ca3b8-1754-4bd8-9f7c-c95c948f5c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:09.141293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ca3b8-1754-4bd8-9f7c-c95c948f5c1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:09.152666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ca3b8-1754-4bd8-9f7c-c95c948f5c1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:00:09.158247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-974ca3b8-1754-4bd8-9f7c-c95c948f5c1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:00:09.158247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:09.158247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:09.158247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:09.158247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:09.158247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:09.158247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:09.158247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:09.158247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:09.158247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:09.158247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:09.158247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:09.158247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:00:09.158247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:00:09.158247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:00:09.158247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:00:09.159198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:00:09.159198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:00:09.159198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:00:09.159198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:00:09.159198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:09.159198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:09.159198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:09.159198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:09.159198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:09.159198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:09.159198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:09.159198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:09.159198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:09.159198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:09.159198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:09.159198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:09.160115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:09.160115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:09.160115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:09.160115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:09.160115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:00:09.160115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:00:09.160115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:00:09.160115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:00:09.160115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:00:09.160115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:00:09.160115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:00:09.160115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:09.160115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 337/667] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 20:00:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:09.160115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ca3b8-1754-4bd8-9f7c-c95c948f5c1f None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:09.222312 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:09.251210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480341d3-7a0a-4a2b-81d1-f2081197288b None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:09.272509 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:09.283926 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ca3b8-1754-4bd8-9f7c-c95c948f5c1f None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:09.292329 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f78a0-3856-497d-9b43-97bec3fb277d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:09.292971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f78a0-3856-497d-9b43-97bec3fb277d None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:09.293251 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f78a0-3856-497d-9b43-97bec3fb277d None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:09.293523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f78a0-3856-497d-9b43-97bec3fb277d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:09.301506 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f78a0-3856-497d-9b43-97bec3fb277d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:09.302706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f78a0-3856-497d-9b43-97bec3fb277d None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:09.315989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-3d3f78a0-3856-497d-9b43-97bec3fb277d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:09.315989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:09.315989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:09.315989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:09.315989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:09.315989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:09.315989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:09.315989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:09.315989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:09.315989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:09.315989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:09.315989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:09.315989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:09.315989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:09.315989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:09.316930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:09.316930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:09.316930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:09.316930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:09.316930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:09.316930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:09.316930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:09.316930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:09.316930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:09.316930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:09.316930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:09.316930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:09.316930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:09.316930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:09.316930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:09.316930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:09.316930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:09.318206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:09.318206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:09.318206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:09.318206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:09.318206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:09.318206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:09.318206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:09.318206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:09.318206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 331/668] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:09.318206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f78a0-3856-497d-9b43-97bec3fb277d None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:09.327607 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb18098e-3ed1-4b9f-99e6-f932b7ecf0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:09.328220 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18098e-3ed1-4b9f-99e6-f932b7ecf0bb None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:09.328506 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18098e-3ed1-4b9f-99e6-f932b7ecf0bb None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:09.328780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18098e-3ed1-4b9f-99e6-f932b7ecf0bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:09.410427 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb18098e-3ed1-4b9f-99e6-f932b7ecf0bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:09.411802 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb18098e-3ed1-4b9f-99e6-f932b7ecf0bb None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:09.432272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f78a0-3856-497d-9b43-97bec3fb277d None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:09.467052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c794b8-ebe1-42b6-b240-e7562e54ebc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:09.467688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c794b8-ebe1-42b6-b240-e7562e54ebc6 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:09.467971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c794b8-ebe1-42b6-b240-e7562e54ebc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:09.468245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c794b8-ebe1-42b6-b240-e7562e54ebc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:09.470301 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:09.475668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c794b8-ebe1-42b6-b240-e7562e54ebc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:09.477798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c794b8-ebe1-42b6-b240-e7562e54ebc6 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:09.490287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-83c794b8-ebe1-42b6-b240-e7562e54ebc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:09.490287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:09.490287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:09.490287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:09.490287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:09.490287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:09.490287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:09.490287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:09.490287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:09.490287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:09.490287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:09.490287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:09.490287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:09.490287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:09.490287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:09.491242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:09.491242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:09.491242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:09.491242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:09.491242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:09.491242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:09.491242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:09.491242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:09.491242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:09.491242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:09.491242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:09.491242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:09.491242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:09.491242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:09.491242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:09.491242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:09.491242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:09.492133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:09.492133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:09.492133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:09.492133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:09.492133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:09.492133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:09.492133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:09.492133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:09.492133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 332/669] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:09.492133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c794b8-ebe1-42b6-b240-e7562e54ebc6 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:09.506730 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:09.517348 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 338/670] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:00:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:00:09.518198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb18098e-3ed1-4b9f-99e6-f932b7ecf0bb None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:09.599245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c794b8-ebe1-42b6-b240-e7562e54ebc6 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:09.618236 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb18098e-3ed1-4b9f-99e6-f932b7ecf0bb None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:09.634468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a14a2-efab-4a94-bcdd-c0cfc9557326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:09.635077 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a14a2-efab-4a94-bcdd-c0cfc9557326 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:09.635364 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a14a2-efab-4a94-bcdd-c0cfc9557326 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:09.635636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a14a2-efab-4a94-bcdd-c0cfc9557326 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:09.636290 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a14a2-efab-4a94-bcdd-c0cfc9557326 None admin] RBAC: Adding query filter params (name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:09.636593 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a14a2-efab-4a94-bcdd-c0cfc9557326 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:09.638049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a14a2-efab-4a94-bcdd-c0cfc9557326 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:09.646413 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 333/671] 10.222.0.73 () {56 vars in 1103 bytes} [Mon Aug 29 20:00:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:09.647833 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc371bb-0631-4071-b583-9887a6381f37 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:09.648122 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc371bb-0631-4071-b583-9887a6381f37 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:09.648403 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc371bb-0631-4071-b583-9887a6381f37 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:09.649638 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 334/672] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:09.650351 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc371bb-0631-4071-b583-9887a6381f37 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:09.658459 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714332b5-8a8b-4fcd-9980-2d49dc93aa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:09.659073 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714332b5-8a8b-4fcd-9980-2d49dc93aa94 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:09.659353 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714332b5-8a8b-4fcd-9980-2d49dc93aa94 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:09.659626 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714332b5-8a8b-4fcd-9980-2d49dc93aa94 None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:09.660315 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714332b5-8a8b-4fcd-9980-2d49dc93aa94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:09.660672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714332b5-8a8b-4fcd-9980-2d49dc93aa94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:09.662089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714332b5-8a8b-4fcd-9980-2d49dc93aa94 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:09.674762 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 339/673] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:00:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:09.675575 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714332b5-8a8b-4fcd-9980-2d49dc93aa94 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:09.713218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:09.726592 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:09.760351 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc371bb-0631-4071-b583-9887a6381f37 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:09.777723 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714332b5-8a8b-4fcd-9980-2d49dc93aa94 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:09.795011 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8dc6ab-2c1c-49e7-a0b6-2be26aafc38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:09.795524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8dc6ab-2c1c-49e7-a0b6-2be26aafc38d None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:09.795806 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8dc6ab-2c1c-49e7-a0b6-2be26aafc38d None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:09.796071 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8dc6ab-2c1c-49e7-a0b6-2be26aafc38d None admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:09.796752 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8dc6ab-2c1c-49e7-a0b6-2be26aafc38d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:09.797051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8dc6ab-2c1c-49e7-a0b6-2be26aafc38d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:09.798491 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8dc6ab-2c1c-49e7-a0b6-2be26aafc38d None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:09.810643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 335/674] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:00:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:09.811460 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8dc6ab-2c1c-49e7-a0b6-2be26aafc38d None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:09.817188 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c22d49-d465-445f-8e44-f1284fd312ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:09.817856 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c22d49-d465-445f-8e44-f1284fd312ef None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:09.818137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c22d49-d465-445f-8e44-f1284fd312ef None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:09.818406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c22d49-d465-445f-8e44-f1284fd312ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:09.829381 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c22d49-d465-445f-8e44-f1284fd312ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:09.830940 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c22d49-d465-445f-8e44-f1284fd312ef None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:09.846025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-91c22d49-d465-445f-8e44-f1284fd312ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:09.846025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:09.846025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:09.846025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:09.846025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:09.846025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:09.846025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:09.846025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:09.846025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:09.846025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:09.846025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:09.846025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:09.846025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:09.846025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:09.846025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:09.846025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:09.847061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:09.847061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:09.847061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:09.847061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:09.847061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:09.847061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:09.847061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:09.847061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:09.847061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:09.847061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:09.847061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:09.847061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:09.847061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:09.847061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:09.847061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:09.847061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:09.847061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:09.847943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:09.847943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:09.847943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:09.847943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 340/675] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:09.848409 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ebf6e-f157-4250-9285-a20a466ad5e2 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:09.848620 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ebf6e-f157-4250-9285-a20a466ad5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:09.848963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ebf6e-f157-4250-9285-a20a466ad5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:09.867401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-c65ebf6e-f157-4250-9285-a20a466ad5e2 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:09.868623 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c65ebf6e-f157-4250-9285-a20a466ad5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:09.916440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8dc6ab-2c1c-49e7-a0b6-2be26aafc38d None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:09.933398 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:09.938651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 341/676] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:00:09.939326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ebf6e-f157-4250-9285-a20a466ad5e2 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:09.961381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce6981-7c5d-4f4d-bb83-5f09cf902906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:09.962057 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce6981-7c5d-4f4d-bb83-5f09cf902906 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:09.962337 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce6981-7c5d-4f4d-bb83-5f09cf902906 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:09.962612 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce6981-7c5d-4f4d-bb83-5f09cf902906 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:09.969844 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:09.976516 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce6981-7c5d-4f4d-bb83-5f09cf902906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:09.977990 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce6981-7c5d-4f4d-bb83-5f09cf902906 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:09.998308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-ebce6981-7c5d-4f4d-bb83-5f09cf902906 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:09.998308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:09.998308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:09.998308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:09.998308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:09.998308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:09.998308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:09.998308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:09.998308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:09.998308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:09.998308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:09.998308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:09.998308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:09.998308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:09.998308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:09.998308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:09.999750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:09.999750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:09.999750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:09.999750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:09.999750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:09.999750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:09.999750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:09.999750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:09.999750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:09.999750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:09.999750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:09.999750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:09.999750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:09.999750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:09.999750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:09.999750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:09.999750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:10.000838 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:10.000838 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:10.000838 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:10.002228 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 336/677] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:10.003249 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce6981-7c5d-4f4d-bb83-5f09cf902906 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:10.047550 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ebf6e-f157-4250-9285-a20a466ad5e2 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:10.086231 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2766ad7-dd75-4f0d-9769-0d93424e61a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:10.086843 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2766ad7-dd75-4f0d-9769-0d93424e61a0 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:10.087120 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2766ad7-dd75-4f0d-9769-0d93424e61a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:10.087393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2766ad7-dd75-4f0d-9769-0d93424e61a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:10.088408 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2766ad7-dd75-4f0d-9769-0d93424e61a0 None admin] RBAC: Adding query filter params (role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=103b64b4b172467692d02851b2955b81) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:10.088848 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2766ad7-dd75-4f0d-9769-0d93424e61a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:10.090333 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2766ad7-dd75-4f0d-9769-0d93424e61a0 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:10.097574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 342/678] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:00:09 2022] GET /identity/v3/role_assignments?user.id=103b64b4b172467692d02851b2955b81&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:10.098432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2766ad7-dd75-4f0d-9769-0d93424e61a0 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:10.128449 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce6981-7c5d-4f4d-bb83-5f09cf902906 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:10.181847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da7154-17f6-469f-933e-53ac967dc04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:10.183141 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da7154-17f6-469f-933e-53ac967dc04a None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:10.183662 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da7154-17f6-469f-933e-53ac967dc04a None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:10.184429 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da7154-17f6-469f-933e-53ac967dc04a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:10.186213 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da7154-17f6-469f-933e-53ac967dc04a None admin] RBAC: Adding query filter params (role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=4a0b2ca8d14a47239a26912e58348f5a) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:10.187072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da7154-17f6-469f-933e-53ac967dc04a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:10.189452 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da7154-17f6-469f-933e-53ac967dc04a None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:10.198561 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 337/679] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:00:10 2022] GET /identity/v3/role_assignments?user.id=4a0b2ca8d14a47239a26912e58348f5a&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:10.199951 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da7154-17f6-469f-933e-53ac967dc04a None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:10.253778 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:10.265140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:10.274069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2766ad7-dd75-4f0d-9769-0d93424e61a0 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:10.325968 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45c9c9-9c9c-4090-a890-309ff2917785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:10.327211 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45c9c9-9c9c-4090-a890-309ff2917785 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:10.328095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45c9c9-9c9c-4090-a890-309ff2917785 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:10.328578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45c9c9-9c9c-4090-a890-309ff2917785 None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:10.329264 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45c9c9-9c9c-4090-a890-309ff2917785 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:10.329570 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45c9c9-9c9c-4090-a890-309ff2917785 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:10.331623 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45c9c9-9c9c-4090-a890-309ff2917785 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:10.348245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 343/680] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:10.349066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45c9c9-9c9c-4090-a890-309ff2917785 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:10.354355 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da7154-17f6-469f-933e-53ac967dc04a None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:10.397503 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bfbe2-9370-4a7b-ae5d-a8e8367640e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:10.398150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bfbe2-9370-4a7b-ae5d-a8e8367640e9 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:10.398431 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bfbe2-9370-4a7b-ae5d-a8e8367640e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:10.399129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bfbe2-9370-4a7b-ae5d-a8e8367640e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:10.399757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bfbe2-9370-4a7b-ae5d-a8e8367640e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:10.399757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bfbe2-9370-4a7b-ae5d-a8e8367640e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:10.401006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bfbe2-9370-4a7b-ae5d-a8e8367640e9 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:10.416836 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 338/681] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:10.417797 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bfbe2-9370-4a7b-ae5d-a8e8367640e9 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:10.472858 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45c9c9-9c9c-4090-a890-309ff2917785 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:10.510829 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00539833-4a2a-4680-a23b-12fa6cfd51a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:10.511654 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00539833-4a2a-4680-a23b-12fa6cfd51a3 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:10.511846 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00539833-4a2a-4680-a23b-12fa6cfd51a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:10.512131 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00539833-4a2a-4680-a23b-12fa6cfd51a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:10.524365 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00539833-4a2a-4680-a23b-12fa6cfd51a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:00:10.529991 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-00539833-4a2a-4680-a23b-12fa6cfd51a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:00:10.529991 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:10.529991 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:10.529991 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:10.529991 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:10.529991 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:10.529991 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:10.529991 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:10.529991 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:10.529991 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:10.529991 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:10.529991 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:10.529991 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:00:10.529991 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:00:10.529991 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:00:10.529991 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:00:10.530942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:00:10.530942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:00:10.530942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:00:10.530942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:00:10.530942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:10.530942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:10.530942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:10.530942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:10.530942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:10.530942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:10.530942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:10.530942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:10.530942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:10.530942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:10.530942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:10.530942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:10.531932 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:10.531932 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:10.531932 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:10.531932 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:10.531932 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:00:10.531932 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:00:10.531932 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:00:10.531932 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:00:10.531932 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:00:10.531932 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:00:10.531932 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:00:10.531932 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:10.532578 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:10.534086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 344/682] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 20:00:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:10.534086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00539833-4a2a-4680-a23b-12fa6cfd51a3 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:10.540911 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bfbe2-9370-4a7b-ae5d-a8e8367640e9 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:10.561036 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:10.579964 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac7bf0-f5c8-465f-8bbb-31da250c6c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:10.580962 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac7bf0-f5c8-465f-8bbb-31da250c6c1c None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:10.581469 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac7bf0-f5c8-465f-8bbb-31da250c6c1c None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:10.581984 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac7bf0-f5c8-465f-8bbb-31da250c6c1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:10.589471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac7bf0-f5c8-465f-8bbb-31da250c6c1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:10.591756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac7bf0-f5c8-465f-8bbb-31da250c6c1c None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:10.603494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-e5ac7bf0-f5c8-465f-8bbb-31da250c6c1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:10.603494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:10.603494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:10.603494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:10.603494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:10.603494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:10.603494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:10.603494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:10.603494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:10.603494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:10.603494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:10.603494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:10.603494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:10.603494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:10.603494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:10.604421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:10.604421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:10.604421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:10.604421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:10.604421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:10.604421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:10.604421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:10.604421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:10.604421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:10.604421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:10.604421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:10.604421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:10.604421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:10.604421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:10.604421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:10.604421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:10.604421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:10.605318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:10.605318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:10.605318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:10.605318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:10.605318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:10.605318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:10.605318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:10.605318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:10.608515 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 339/683] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:10.609732 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac7bf0-f5c8-465f-8bbb-31da250c6c1c None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:10.663395 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00539833-4a2a-4680-a23b-12fa6cfd51a3 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:10.701483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51829d92-3ea2-4f05-8cf9-40cdede2f8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:10.702322 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51829d92-3ea2-4f05-8cf9-40cdede2f8f2 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:10.702451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51829d92-3ea2-4f05-8cf9-40cdede2f8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:10.702664 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51829d92-3ea2-4f05-8cf9-40cdede2f8f2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:10.710269 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51829d92-3ea2-4f05-8cf9-40cdede2f8f2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:10.711722 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51829d92-3ea2-4f05-8cf9-40cdede2f8f2 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:10.725512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-51829d92-3ea2-4f05-8cf9-40cdede2f8f2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:00:10.725512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:10.725512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:10.725512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:10.725512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:10.725512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:10.725512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:10.725512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:10.725512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:10.725512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:10.725512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:10.725512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:10.725512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:00:10.725512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:00:10.725512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:00:10.725512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:00:10.726542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:10.726542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:10.726542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:10.726542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:10.726542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:10.726542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:10.726542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:10.726542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:10.726542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:10.726542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:10.726542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:10.726542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:10.726542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:10.726542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:10.726542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:10.726542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:10.726542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:10.726542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:10.727494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:10.727494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:10.727494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:00:10.727494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:00:10.727494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:00:10.727494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:00:10.727494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:00:10.727494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:00:10.727494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:00:10.727494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:10.731509 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 345/684] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 20:00:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:10.732915 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac7bf0-f5c8-465f-8bbb-31da250c6c1c None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:10.733036 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51829d92-3ea2-4f05-8cf9-40cdede2f8f2 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:10.781458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251a43a-d68a-4ac4-a761-0f482ec33981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:10.783025 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251a43a-d68a-4ac4-a761-0f482ec33981 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:10.783596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251a43a-d68a-4ac4-a761-0f482ec33981 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:10.784712 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251a43a-d68a-4ac4-a761-0f482ec33981 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:10.786360 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251a43a-d68a-4ac4-a761-0f482ec33981 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:10.787038 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251a43a-d68a-4ac4-a761-0f482ec33981 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:10.788998 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251a43a-d68a-4ac4-a761-0f482ec33981 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:10.804189 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 340/685] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 20:00:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:10.805686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251a43a-d68a-4ac4-a761-0f482ec33981 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:10.827785 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:10.838935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:10.873552 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51829d92-3ea2-4f05-8cf9-40cdede2f8f2 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:10.917422 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ed23e-3939-4974-9937-38d6d1dfa989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:10.918181 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ed23e-3939-4974-9937-38d6d1dfa989 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:10.918464 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ed23e-3939-4974-9937-38d6d1dfa989 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:10.918760 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ed23e-3939-4974-9937-38d6d1dfa989 None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:10.920395 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ed23e-3939-4974-9937-38d6d1dfa989 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:10.920704 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ed23e-3939-4974-9937-38d6d1dfa989 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:10.922319 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ed23e-3939-4974-9937-38d6d1dfa989 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:10.938163 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 346/686] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:00:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:10.939038 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ed23e-3939-4974-9937-38d6d1dfa989 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:10.961693 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251a43a-d68a-4ac4-a761-0f482ec33981 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:11.008458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff09ec0-c64e-42c4-be90-1cf35e61c606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:11.009554 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff09ec0-c64e-42c4-be90-1cf35e61c606 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:11.010091 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff09ec0-c64e-42c4-be90-1cf35e61c606 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:11.010833 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff09ec0-c64e-42c4-be90-1cf35e61c606 None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:11.012007 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff09ec0-c64e-42c4-be90-1cf35e61c606 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:11.012516 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff09ec0-c64e-42c4-be90-1cf35e61c606 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:11.014894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff09ec0-c64e-42c4-be90-1cf35e61c606 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:11.038156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 341/687] 10.222.0.73 () {56 vars in 1135 bytes} [Mon Aug 29 20:00:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:11.039823 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff09ec0-c64e-42c4-be90-1cf35e61c606 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:11.052620 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ed23e-3939-4974-9937-38d6d1dfa989 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:11.094520 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33333ae7-c5d8-4e21-bf33-ff76bc609bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:11.095327 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33333ae7-c5d8-4e21-bf33-ff76bc609bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:11.095716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33333ae7-c5d8-4e21-bf33-ff76bc609bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:11.096053 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33333ae7-c5d8-4e21-bf33-ff76bc609bb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:11.101499 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:11.112374 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33333ae7-c5d8-4e21-bf33-ff76bc609bb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:11.114130 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33333ae7-c5d8-4e21-bf33-ff76bc609bb5 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:11.134908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-33333ae7-c5d8-4e21-bf33-ff76bc609bb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:11.134908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:11.134908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:11.134908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:11.134908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:11.134908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:11.134908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:11.134908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:11.134908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:11.134908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:11.134908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:11.134908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:11.134908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:11.134908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:11.134908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:11.134908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:11.135915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:11.135915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:11.135915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:11.135915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:11.135915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:11.135915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:11.135915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:11.135915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:11.135915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:11.135915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:11.135915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:11.135915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:11.135915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:11.135915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:11.135915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:11.135915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:11.135915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:11.136825 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:11.136825 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:11.136825 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:11.139612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 347/688] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:11.140619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33333ae7-c5d8-4e21-bf33-ff76bc609bb5 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:11.148159 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:11.180365 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff09ec0-c64e-42c4-be90-1cf35e61c606 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:11.219156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a2c2d-1387-4f46-9109-2837de945cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:11.219784 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a2c2d-1387-4f46-9109-2837de945cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:11.220193 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a2c2d-1387-4f46-9109-2837de945cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:11.220633 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a2c2d-1387-4f46-9109-2837de945cd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:11.221435 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a2c2d-1387-4f46-9109-2837de945cd8 None admin] RBAC: Adding query filter params (role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=64ba58d774674ed7b233ad6416db068a) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:11.222052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a2c2d-1387-4f46-9109-2837de945cd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:11.223726 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a2c2d-1387-4f46-9109-2837de945cd8 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:11.232394 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 342/689] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:00:11 2022] GET /identity/v3/role_assignments?user.id=64ba58d774674ed7b233ad6416db068a&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:11.233671 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a2c2d-1387-4f46-9109-2837de945cd8 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:11.328277 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33333ae7-c5d8-4e21-bf33-ff76bc609bb5 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:11.366242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a2c2d-1387-4f46-9109-2837de945cd8 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:11.410124 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890fac4-1ec4-41b8-9e08-6da67c06f793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:11.410994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890fac4-1ec4-41b8-9e08-6da67c06f793 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:11.411350 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890fac4-1ec4-41b8-9e08-6da67c06f793 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:11.411625 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92acd1d0-831f-40e3-a683-ceed13d2848a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:11.411765 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890fac4-1ec4-41b8-9e08-6da67c06f793 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:11.412542 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92acd1d0-831f-40e3-a683-ceed13d2848a None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:11.412990 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92acd1d0-831f-40e3-a683-ceed13d2848a None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:11.413449 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92acd1d0-831f-40e3-a683-ceed13d2848a None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:11.414300 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92acd1d0-831f-40e3-a683-ceed13d2848a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:11.414847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92acd1d0-831f-40e3-a683-ceed13d2848a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:11.416444 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92acd1d0-831f-40e3-a683-ceed13d2848a None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:11.420100 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:11.434499 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 343/690] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:11.448845 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92acd1d0-831f-40e3-a683-ceed13d2848a None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:11.495659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:11.513646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890fac4-1ec4-41b8-9e08-6da67c06f793 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:11.515145 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890fac4-1ec4-41b8-9e08-6da67c06f793 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:11.566434 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92acd1d0-831f-40e3-a683-ceed13d2848a None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:11.611810 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206ad24-3f7f-485c-962c-4c43b969108f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:11.612455 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206ad24-3f7f-485c-962c-4c43b969108f None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:11.612734 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206ad24-3f7f-485c-962c-4c43b969108f None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:11.613003 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206ad24-3f7f-485c-962c-4c43b969108f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:11.623424 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 348/691] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:00:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:00:11.626905 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206ad24-3f7f-485c-962c-4c43b969108f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:11.628400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206ad24-3f7f-485c-962c-4c43b969108f None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:11.629793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890fac4-1ec4-41b8-9e08-6da67c06f793 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:11.649513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-9206ad24-3f7f-485c-962c-4c43b969108f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:00:11.649513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:11.649513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:11.649513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:11.649513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:11.649513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:11.649513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:11.649513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:11.649513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:11.649513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:11.649513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:11.649513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:11.649513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:00:11.649513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:00:11.649513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:00:11.649513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:00:11.650665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:11.650665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:11.650665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:11.650665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:11.650665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:11.650665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:11.650665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:11.650665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:11.650665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:11.650665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:11.650665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:11.650665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:11.650665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:11.650665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:11.650665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:11.650665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:11.650665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:11.650665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:11.651608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:11.651608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:11.651608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:00:11.651608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:00:11.651608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:00:11.651608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:00:11.651608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:00:11.651608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:00:11.651608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:00:11.651608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:11.651608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 344/692] 10.222.0.73 () {56 vars in 1093 bytes} [Mon Aug 29 20:00:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:11.657419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206ad24-3f7f-485c-962c-4c43b969108f None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:11.730415 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:11.734091 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:11.745027 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890fac4-1ec4-41b8-9e08-6da67c06f793 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:11.781984 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206ad24-3f7f-485c-962c-4c43b969108f None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:11.783419 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a772ab4-4778-4300-ae26-ae75f1d89108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:11.784136 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a772ab4-4778-4300-ae26-ae75f1d89108 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:11.784475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a772ab4-4778-4300-ae26-ae75f1d89108 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:11.784816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a772ab4-4778-4300-ae26-ae75f1d89108 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:11.796830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a772ab4-4778-4300-ae26-ae75f1d89108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:11.798446 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a772ab4-4778-4300-ae26-ae75f1d89108 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:11.815874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-6a772ab4-4778-4300-ae26-ae75f1d89108 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:11.815874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:11.815874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:11.815874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:11.815874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:11.815874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:11.815874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:11.815874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:11.815874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:11.815874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:11.815874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:11.815874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:11.815874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:11.815874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:11.815874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:11.815874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:11.816880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:11.816880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:11.816880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:11.816880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:11.816880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:11.816880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:11.816880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:11.816880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:11.816880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:11.816880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:11.816880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:11.816880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:11.816880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:11.816880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:11.816880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:11.816880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:11.816880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:11.817801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:11.817801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:11.817801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:11.821859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 349/693] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:11.822748 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a772ab4-4778-4300-ae26-ae75f1d89108 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:11.830492 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a92e22-fd35-4145-935b-b82742d3daeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:11.831123 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a92e22-fd35-4145-935b-b82742d3daeb None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:11.831460 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a92e22-fd35-4145-935b-b82742d3daeb None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:11.831737 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a92e22-fd35-4145-935b-b82742d3daeb None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:11.832529 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a92e22-fd35-4145-935b-b82742d3daeb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:11.832836 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a92e22-fd35-4145-935b-b82742d3daeb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:11.835107 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a92e22-fd35-4145-935b-b82742d3daeb None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:11.870784 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 345/694] 10.222.0.73 () {56 vars in 1133 bytes} [Mon Aug 29 20:00:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:11.876870 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a92e22-fd35-4145-935b-b82742d3daeb None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:12.004957 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a772ab4-4778-4300-ae26-ae75f1d89108 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:12.048813 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446927c-6c01-4b8f-bb12-06524d47aa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:12.051140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446927c-6c01-4b8f-bb12-06524d47aa55 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:12.051440 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446927c-6c01-4b8f-bb12-06524d47aa55 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:12.051721 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446927c-6c01-4b8f-bb12-06524d47aa55 None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:12.052377 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446927c-6c01-4b8f-bb12-06524d47aa55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:12.052682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446927c-6c01-4b8f-bb12-06524d47aa55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:12.057401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446927c-6c01-4b8f-bb12-06524d47aa55 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:12.072823 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a92e22-fd35-4145-935b-b82742d3daeb None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:12.084124 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 350/695] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:12.085255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:12.092299 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446927c-6c01-4b8f-bb12-06524d47aa55 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:12.115357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:12.167557 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10869b0f-29a9-4eb1-a8de-158dc8e54e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:12.168062 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10869b0f-29a9-4eb1-a8de-158dc8e54e8b None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:12.168339 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10869b0f-29a9-4eb1-a8de-158dc8e54e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:12.168609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10869b0f-29a9-4eb1-a8de-158dc8e54e8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:12.218984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446927c-6c01-4b8f-bb12-06524d47aa55 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:12.256201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341672b6-9842-4738-a889-25695b87cdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:12.257189 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341672b6-9842-4738-a889-25695b87cdeb None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:12.257679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341672b6-9842-4738-a889-25695b87cdeb None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:12.258212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341672b6-9842-4738-a889-25695b87cdeb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:12.267900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341672b6-9842-4738-a889-25695b87cdeb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:12.270544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341672b6-9842-4738-a889-25695b87cdeb None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:12.272274 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10869b0f-29a9-4eb1-a8de-158dc8e54e8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:12.273882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10869b0f-29a9-4eb1-a8de-158dc8e54e8b None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:12.286960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-341672b6-9842-4738-a889-25695b87cdeb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:12.286960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:12.286960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:12.286960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:12.286960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:12.286960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:12.286960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:12.286960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:12.286960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:12.286960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:12.286960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:12.286960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:12.286960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:12.286960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:12.286960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:12.287924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:12.287924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:12.287924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:12.287924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:12.287924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:12.287924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:12.287924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:12.287924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:12.287924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:12.287924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:12.287924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:12.287924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:12.287924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:12.287924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:12.287924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:12.287924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:12.287924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:12.288817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:12.288817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:12.288817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:12.288817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:12.288817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:12.288817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:12.288817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:12.288817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:12.291176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 351/696] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:12.293166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50b787-5fcb-4680-90df-5f1603563235 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:12.293625 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50b787-5fcb-4680-90df-5f1603563235 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:12.294378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50b787-5fcb-4680-90df-5f1603563235 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:12.296673 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 352/697] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:12.297556 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50b787-5fcb-4680-90df-5f1603563235 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:12.367513 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:12.417911 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 346/698] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:00:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 20:00:12.420727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107951d7-aa3d-4353-8d45-8d8fe6addd3f None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:12.421012 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107951d7-aa3d-4353-8d45-8d8fe6addd3f None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:12.421293 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107951d7-aa3d-4353-8d45-8d8fe6addd3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:12.426782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50b787-5fcb-4680-90df-5f1603563235 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:12.440844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-107951d7-aa3d-4353-8d45-8d8fe6addd3f None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:12.442076 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-107951d7-aa3d-4353-8d45-8d8fe6addd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:12.453203 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:12.467334 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228de62-9067-4e1b-8b5d-49d9e31d5489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:12.467944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228de62-9067-4e1b-8b5d-49d9e31d5489 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:12.468223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228de62-9067-4e1b-8b5d-49d9e31d5489 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:12.468499 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228de62-9067-4e1b-8b5d-49d9e31d5489 None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:12.469185 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a228de62-9067-4e1b-8b5d-49d9e31d5489 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:12.469493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a228de62-9067-4e1b-8b5d-49d9e31d5489 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:12.470969 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a228de62-9067-4e1b-8b5d-49d9e31d5489 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:12.483456 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 353/699] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:00:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:12.484497 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228de62-9067-4e1b-8b5d-49d9e31d5489 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:12.542313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 347/700] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:00:12.542313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519170b7-24d4-455a-a770-7abdfb4127ff None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:12.542313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519170b7-24d4-455a-a770-7abdfb4127ff None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:12.542313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519170b7-24d4-455a-a770-7abdfb4127ff None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:12.542313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 348/701] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:12.542313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519170b7-24d4-455a-a770-7abdfb4127ff None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:12.603306 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228de62-9067-4e1b-8b5d-49d9e31d5489 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:12.624614 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:12.641089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73ab47-6ce8-4f57-a316-7caaba385398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:12.641732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73ab47-6ce8-4f57-a316-7caaba385398 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:12.642036 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73ab47-6ce8-4f57-a316-7caaba385398 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:12.642307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73ab47-6ce8-4f57-a316-7caaba385398 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:12.654352 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73ab47-6ce8-4f57-a316-7caaba385398 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:12.655884 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73ab47-6ce8-4f57-a316-7caaba385398 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:12.675201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-aa73ab47-6ce8-4f57-a316-7caaba385398 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:12.675201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:12.675201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:12.675201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:12.675201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:12.675201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:12.675201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:12.675201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:12.675201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:12.675201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:12.675201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:12.675201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:12.675201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:12.675201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:12.675201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:12.675201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:12.676173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:12.676173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:12.676173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:12.676173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:12.676173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:12.676173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:12.676173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:12.676173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:12.676173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:12.676173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:12.676173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:12.676173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:12.676173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:12.676173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:12.676173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:12.676173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:12.676173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:12.677066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:12.677066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:12.677066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:12.679298 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 354/702] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:12.680637 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfc45c-f27a-4647-8049-9d58d6180b22 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:12.681125 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfc45c-f27a-4647-8049-9d58d6180b22 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:12.681680 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfc45c-f27a-4647-8049-9d58d6180b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:12.686417 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519170b7-24d4-455a-a770-7abdfb4127ff None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:12.702539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-90bfc45c-f27a-4647-8049-9d58d6180b22 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:12.704093 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-90bfc45c-f27a-4647-8049-9d58d6180b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:12.733666 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fa946-7c8a-4b7c-9956-da3256d50a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:12.734587 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fa946-7c8a-4b7c-9956-da3256d50a1f None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:12.734872 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fa946-7c8a-4b7c-9956-da3256d50a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:12.735151 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fa946-7c8a-4b7c-9956-da3256d50a1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:12.736332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7fa946-7c8a-4b7c-9956-da3256d50a1f None admin] RBAC: Adding query filter params (role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=b92e30da8f394f3abc19b9e86ef126e5) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:12.736647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7fa946-7c8a-4b7c-9956-da3256d50a1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:12.738724 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7fa946-7c8a-4b7c-9956-da3256d50a1f None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:12.747258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 349/703] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:00:12 2022] GET /identity/v3/role_assignments?user.id=b92e30da8f394f3abc19b9e86ef126e5&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:12.748712 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fa946-7c8a-4b7c-9956-da3256d50a1f None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:12.767468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:12.789483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 355/704] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:00:12.886760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fa946-7c8a-4b7c-9956-da3256d50a1f None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:12.926288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e420473-5e40-4b96-a34f-d49973e93eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:12.927267 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e420473-5e40-4b96-a34f-d49973e93eec None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:12.927793 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e420473-5e40-4b96-a34f-d49973e93eec None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:12.928294 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e420473-5e40-4b96-a34f-d49973e93eec None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:12.929224 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e420473-5e40-4b96-a34f-d49973e93eec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:12.929739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e420473-5e40-4b96-a34f-d49973e93eec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:12.930173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfc45c-f27a-4647-8049-9d58d6180b22 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:12.931547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e420473-5e40-4b96-a34f-d49973e93eec None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:12.945669 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 350/705] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:12.953586 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e420473-5e40-4b96-a34f-d49973e93eec None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:13.044760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:13.069598 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfc45c-f27a-4647-8049-9d58d6180b22 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:13.071360 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:13.094331 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e420473-5e40-4b96-a34f-d49973e93eec None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:13.111212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3dc4c-9652-485f-bda4-d12de0c730fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:13.112144 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3dc4c-9652-485f-bda4-d12de0c730fe None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:13.112426 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3dc4c-9652-485f-bda4-d12de0c730fe None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:13.112696 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3dc4c-9652-485f-bda4-d12de0c730fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:13.127398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3dc4c-9652-485f-bda4-d12de0c730fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:13.129183 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3dc4c-9652-485f-bda4-d12de0c730fe None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:13.142858 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d0aa5-a7cc-4d1a-8a67-ec2d2e9b11bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:13.144313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d0aa5-a7cc-4d1a-8a67-ec2d2e9b11bd None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:13.145034 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d0aa5-a7cc-4d1a-8a67-ec2d2e9b11bd None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:13.145762 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d0aa5-a7cc-4d1a-8a67-ec2d2e9b11bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:13.147679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-87e3dc4c-9652-485f-bda4-d12de0c730fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:13.147679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:13.147679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:13.147679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:13.147679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:13.147679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:13.147679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:13.147679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:13.147679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:13.147679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:13.147679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:13.147679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:13.147679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:13.147679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:13.147679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:13.147679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:13.148651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:13.148651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:13.148651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:13.148651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:13.148651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:13.148651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:13.148651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:13.148651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:13.148651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:13.148651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:13.148651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:13.148651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:13.148651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:13.148651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:13.148651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:13.148651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:13.148651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:13.149621 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:13.149621 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:13.149621 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:13.149621 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 356/706] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:00:13.149621 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3dc4c-9652-485f-bda4-d12de0c730fe None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:13.157405 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d0aa5-a7cc-4d1a-8a67-ec2d2e9b11bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:13.158915 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d0aa5-a7cc-4d1a-8a67-ec2d2e9b11bd None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:13.171788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-bf5d0aa5-a7cc-4d1a-8a67-ec2d2e9b11bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:13.171788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:13.171788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:13.171788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:13.171788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:13.171788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:13.171788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:13.171788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:13.171788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:13.171788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:13.171788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:13.171788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:13.171788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:13.171788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:13.171788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:13.172774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:13.172774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:13.172774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:13.172774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:13.172774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:13.172774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:13.172774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:13.172774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:13.172774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:13.172774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:13.172774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:13.172774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:13.172774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:13.172774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:13.172774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:13.172774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:13.172774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:13.173700 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:13.173700 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:13.173700 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:13.173700 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:13.173700 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:13.173700 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:13.173700 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:13.173700 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:13.173700 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 351/707] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:13.175051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d0aa5-a7cc-4d1a-8a67-ec2d2e9b11bd None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:13.282168 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3dc4c-9652-485f-bda4-d12de0c730fe None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:13.299731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d0aa5-a7cc-4d1a-8a67-ec2d2e9b11bd None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:13.328687 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749e843-e716-44ef-aa50-9fca75842bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:13.329584 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749e843-e716-44ef-aa50-9fca75842bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:13.330327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:13.331619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749e843-e716-44ef-aa50-9fca75842bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:13.331900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749e843-e716-44ef-aa50-9fca75842bc1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:13.332508 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749e843-e716-44ef-aa50-9fca75842bc1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:13.335360 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749e843-e716-44ef-aa50-9fca75842bc1 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:13.338980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402db76a-5bac-4584-85d4-28ca0e6a31c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:13.339664 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402db76a-5bac-4584-85d4-28ca0e6a31c4 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:13.339964 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402db76a-5bac-4584-85d4-28ca0e6a31c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:13.340216 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402db76a-5bac-4584-85d4-28ca0e6a31c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:13.340850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402db76a-5bac-4584-85d4-28ca0e6a31c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:13.341216 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402db76a-5bac-4584-85d4-28ca0e6a31c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:13.342758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402db76a-5bac-4584-85d4-28ca0e6a31c4 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:13.349282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-2749e843-e716-44ef-aa50-9fca75842bc1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:00:13.349282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:13.349282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:00:13.349282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:00:13.349282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:00:13.349282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:00:13.349282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:00:13.349282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:13.349282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:00:13.349282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:13.349282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:13.349282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:13.349282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:13.349282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:13.349282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:13.349282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:13.349282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:13.350401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:13.350401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:13.350401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:13.350401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:13.350401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:13.350401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:13.350401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:00:13.350401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:00:13.350401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:00:13.350401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:00:13.350401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:13.350401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:13.350401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:13.350401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:13.350401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:13.350401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:13.351251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:13.351251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:13.351251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:13.351251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:13.351251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:13.351251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:13.351251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:13.351251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:13.351251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:13.351251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:13.351251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:00:13.351251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:13.351251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:00:13.351251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:13.351984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 357/708] 10.222.0.73 () {56 vars in 1129 bytes} [Mon Aug 29 20:00:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:13.353867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749e843-e716-44ef-aa50-9fca75842bc1 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:13.361839 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 352/709] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:13.362696 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402db76a-5bac-4584-85d4-28ca0e6a31c4 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:13.382598 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:13.486701 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749e843-e716-44ef-aa50-9fca75842bc1 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:13.499135 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402db76a-5bac-4584-85d4-28ca0e6a31c4 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:13.532963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c835308-5415-4a3a-864d-3ac5126e75ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:13.533913 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c835308-5415-4a3a-864d-3ac5126e75ed None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:13.534297 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c835308-5415-4a3a-864d-3ac5126e75ed None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:13.535063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c835308-5415-4a3a-864d-3ac5126e75ed None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:13.536133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c835308-5415-4a3a-864d-3ac5126e75ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:13.536544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c835308-5415-4a3a-864d-3ac5126e75ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:13.538412 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c835308-5415-4a3a-864d-3ac5126e75ed None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:13.540695 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4d7ca-b1e9-433d-b02e-11da4928dc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:13.541638 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4d7ca-b1e9-433d-b02e-11da4928dc00 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:13.542101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4d7ca-b1e9-433d-b02e-11da4928dc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:13.542528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4d7ca-b1e9-433d-b02e-11da4928dc00 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:13.543455 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4d7ca-b1e9-433d-b02e-11da4928dc00 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:13.543920 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4d7ca-b1e9-433d-b02e-11da4928dc00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:13.546409 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4d7ca-b1e9-433d-b02e-11da4928dc00 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:13.553662 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 358/710] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:00:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:13.558576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c835308-5415-4a3a-864d-3ac5126e75ed None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:13.561271 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 353/711] 10.222.0.73 () {56 vars in 1124 bytes} [Mon Aug 29 20:00:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:13.562510 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4d7ca-b1e9-433d-b02e-11da4928dc00 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:13.629303 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:13.668419 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:13.694101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c835308-5415-4a3a-864d-3ac5126e75ed None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:13.703016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4d7ca-b1e9-433d-b02e-11da4928dc00 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:13.742174 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773de0f-1c8f-4590-bc97-2952b6e96b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:13.743316 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773de0f-1c8f-4590-bc97-2952b6e96b20 None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:13.743428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645c636-34a4-4651-bbae-4bbf0ff90d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:13.743932 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645c636-34a4-4651-bbae-4bbf0ff90d75 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:13.744155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773de0f-1c8f-4590-bc97-2952b6e96b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:13.744270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645c636-34a4-4651-bbae-4bbf0ff90d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:13.744471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645c636-34a4-4651-bbae-4bbf0ff90d75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:13.744914 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773de0f-1c8f-4590-bc97-2952b6e96b20 None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:13.745193 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645c636-34a4-4651-bbae-4bbf0ff90d75 None admin] RBAC: Adding query filter params (role.id=296ee2c194824485af71618619f82e09, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=95cf77a92a174ff8b0503baf5c7e63be) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:13.745501 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645c636-34a4-4651-bbae-4bbf0ff90d75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:13.746197 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773de0f-1c8f-4590-bc97-2952b6e96b20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:13.746975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645c636-34a4-4651-bbae-4bbf0ff90d75 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:13.748604 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773de0f-1c8f-4590-bc97-2952b6e96b20 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:13.757749 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 354/712] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:00:13 2022] GET /identity/v3/role_assignments?user.id=95cf77a92a174ff8b0503baf5c7e63be&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=296ee2c194824485af71618619f82e09 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:13.757749 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645c636-34a4-4651-bbae-4bbf0ff90d75 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:13.787982 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 359/713] 10.222.0.73 () {60 vars in 1132 bytes} [Mon Aug 29 20:00:13 2022] POST /identity/v3/users => generated 254 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:00:13.789728 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa10e7b-3082-4816-9a9d-20c04692f2cf None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:13.790181 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa10e7b-3082-4816-9a9d-20c04692f2cf None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:13.790647 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa10e7b-3082-4816-9a9d-20c04692f2cf None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:13.792617 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 360/714] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:13.799453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637b6d4-c7aa-47de-8c4a-6174975e8bc5 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:13.799967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637b6d4-c7aa-47de-8c4a-6174975e8bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:13.800448 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637b6d4-c7aa-47de-8c4a-6174975e8bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:13.823384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-8637b6d4-c7aa-47de-8c4a-6174975e8bc5 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:13.825151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-8637b6d4-c7aa-47de-8c4a-6174975e8bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:13.889135 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645c636-34a4-4651-bbae-4bbf0ff90d75 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:13.907847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:13.908790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 361/715] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:00:13.944450 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0674813-6a3a-4794-b084-39ae2c4e4148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:13.945371 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0674813-6a3a-4794-b084-39ae2c4e4148 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:13.945884 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0674813-6a3a-4794-b084-39ae2c4e4148 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:13.946369 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0674813-6a3a-4794-b084-39ae2c4e4148 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:13.947166 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0674813-6a3a-4794-b084-39ae2c4e4148 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:13.947645 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0674813-6a3a-4794-b084-39ae2c4e4148 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:13.949173 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0674813-6a3a-4794-b084-39ae2c4e4148 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:13.959098 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 355/716] 10.222.0.73 () {56 vars in 1124 bytes} [Mon Aug 29 20:00:13 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:14.190943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637b6d4-c7aa-47de-8c4a-6174975e8bc5 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:14.265668 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:14.309265 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637b6d4-c7aa-47de-8c4a-6174975e8bc5 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:14.355100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1d6c5-3693-44a9-9dbc-f2c0a6b3d584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:14.355664 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1d6c5-3693-44a9-9dbc-f2c0a6b3d584 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:14.355942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1d6c5-3693-44a9-9dbc-f2c0a6b3d584 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:14.356212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1d6c5-3693-44a9-9dbc-f2c0a6b3d584 None admin] PATH_INFO: `/v3/services/network` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:14.356913 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1d6c5-3693-44a9-9dbc-f2c0a6b3d584 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:14.358772 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1d6c5-3693-44a9-9dbc-f2c0a6b3d584 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:14.369941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-20e1d6c5-3693-44a9-9dbc-f2c0a6b3d584 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:00:14.369941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:14.369941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:00:14.369941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:00:14.369941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:00:14.369941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:00:14.369941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:00:14.369941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:14.369941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:00:14.369941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:14.369941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:14.369941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:14.369941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:14.369941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:14.369941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:14.369941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:14.369941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:14.370963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:14.370963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:14.370963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:14.370963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:14.370963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:14.370963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:14.370963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:00:14.370963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:00:14.370963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:00:14.370963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:00:14.370963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:14.370963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:14.370963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:14.370963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:14.370963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:14.370963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:14.372215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:14.372215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:14.372215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:14.372215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:14.372215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:14.372215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:14.372215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:14.372215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:14.372215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:14.372215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:14.372215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:00:14.372215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:14.372215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:00:14.372215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:14.372215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 362/717] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:14 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:14.378845 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0674813-6a3a-4794-b084-39ae2c4e4148 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:14.497314 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0674813-6a3a-4794-b084-39ae2c4e4148 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:14.533590 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf21123-2fd8-452f-a8a9-9977c4df2b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:14.534244 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf21123-2fd8-452f-a8a9-9977c4df2b10 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:14.534535 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf21123-2fd8-452f-a8a9-9977c4df2b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:14.534809 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf21123-2fd8-452f-a8a9-9977c4df2b10 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:14.535448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf21123-2fd8-452f-a8a9-9977c4df2b10 None admin] RBAC: Adding query filter params (name=network) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:14.535754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf21123-2fd8-452f-a8a9-9977c4df2b10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:14.537117 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf21123-2fd8-452f-a8a9-9977c4df2b10 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:14.542161 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 356/718] 10.222.0.73 () {56 vars in 1112 bytes} [Mon Aug 29 20:00:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:14.548039 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1d6c5-3693-44a9-9dbc-f2c0a6b3d584 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:14.670143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1d6c5-3693-44a9-9dbc-f2c0a6b3d584 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:14.698315 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:14.706730 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd78bab-4b10-4a5b-9edd-e7f8b52dd0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:14.707350 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd78bab-4b10-4a5b-9edd-e7f8b52dd0b0 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:14.708036 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd78bab-4b10-4a5b-9edd-e7f8b52dd0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:14.708315 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd78bab-4b10-4a5b-9edd-e7f8b52dd0b0 None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:14.708955 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd78bab-4b10-4a5b-9edd-e7f8b52dd0b0 None admin] RBAC: Adding query filter params (type=network) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:14.709612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd78bab-4b10-4a5b-9edd-e7f8b52dd0b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:14.714666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd78bab-4b10-4a5b-9edd-e7f8b52dd0b0 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:14.721974 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 363/719] 10.222.0.73 () {56 vars in 1112 bytes} [Mon Aug 29 20:00:14 2022] GET /identity/v3/services?type=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:15.363851 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356021c3-7296-43e7-b2ac-4e8e19ee78a2 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:15.364348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356021c3-7296-43e7-b2ac-4e8e19ee78a2 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:15.364348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356021c3-7296-43e7-b2ac-4e8e19ee78a2 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:15.365957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 357/720] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:15.371226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee1b4d-77d6-4df8-b459-595006d39098 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:15.371520 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee1b4d-77d6-4df8-b459-595006d39098 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:15.371803 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee1b4d-77d6-4df8-b459-595006d39098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:15.398426 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-d5ee1b4d-77d6-4df8-b459-595006d39098 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:15.399631 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-d5ee1b4d-77d6-4df8-b459-595006d39098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:15.492001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 364/721] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:00:15.898805 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356021c3-7296-43e7-b2ac-4e8e19ee78a2 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:15.951117 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:16.010341 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356021c3-7296-43e7-b2ac-4e8e19ee78a2 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:16.046620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e12d2-3b37-48a3-8a5f-9e227c0c8d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:16.047434 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e12d2-3b37-48a3-8a5f-9e227c0c8d00 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:16.047768 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e12d2-3b37-48a3-8a5f-9e227c0c8d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:16.048040 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e12d2-3b37-48a3-8a5f-9e227c0c8d00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:16.059201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047e12d2-3b37-48a3-8a5f-9e227c0c8d00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:00:16.065205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-047e12d2-3b37-48a3-8a5f-9e227c0c8d00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:00:16.065205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:16.065205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:16.065205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:16.065205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:16.065205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:16.065205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:16.065205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:16.065205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:16.065205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:16.065205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:16.065205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:16.065205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:00:16.065205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:00:16.065205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:00:16.065205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:00:16.066190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:00:16.066190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:00:16.066190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:00:16.066190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:00:16.066190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:16.066190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:16.066190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:16.066190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:16.066190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:16.066190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:16.066190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:16.066190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:16.066190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:16.066190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:16.066190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:16.066190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:16.067052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:16.067052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:16.067052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:16.067052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:16.067052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:00:16.067052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:00:16.067052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:00:16.067052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:00:16.067052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:00:16.067052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:00:16.067052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:00:16.067052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:16.067052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 358/722] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 20:00:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:16.072926 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee1b4d-77d6-4df8-b459-595006d39098 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:16.229647 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee1b4d-77d6-4df8-b459-595006d39098 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:16.256069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a2bac-b9da-4db5-9e32-acfc952e79a3 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:16.256401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a2bac-b9da-4db5-9e32-acfc952e79a3 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:16.256624 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a2bac-b9da-4db5-9e32-acfc952e79a3 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:16.261747 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 359/723] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:16.267271 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25464f-fae7-421e-a388-abfe17e67eef None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:16.267552 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25464f-fae7-421e-a388-abfe17e67eef None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:16.267820 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25464f-fae7-421e-a388-abfe17e67eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:16.284144 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:16.294076 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-5a25464f-fae7-421e-a388-abfe17e67eef None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:16.295824 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-5a25464f-fae7-421e-a388-abfe17e67eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:16.311032 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b1314-31dc-434a-a533-abda1f967260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:16.311655 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b1314-31dc-434a-a533-abda1f967260 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:16.311935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b1314-31dc-434a-a533-abda1f967260 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:16.312274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b1314-31dc-434a-a533-abda1f967260 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:16.312958 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b1314-31dc-434a-a533-abda1f967260 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:16.313261 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b1314-31dc-434a-a533-abda1f967260 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:16.314851 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b1314-31dc-434a-a533-abda1f967260 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:16.330783 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 365/724] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 20:00:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:16.343443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b1314-31dc-434a-a533-abda1f967260 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:16.408510 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 360/725] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:00:16.410980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7666d8c-49a6-47d1-a969-c05e7ab56523 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:16.411606 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7666d8c-49a6-47d1-a969-c05e7ab56523 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:16.412360 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7666d8c-49a6-47d1-a969-c05e7ab56523 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:16.414309 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 361/726] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:16.420245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e3cc2-f66b-40cf-a0f0-e63dbe6bccc4 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:16.420855 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e3cc2-f66b-40cf-a0f0-e63dbe6bccc4 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:16.421438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e3cc2-f66b-40cf-a0f0-e63dbe6bccc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:16.439616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-f21e3cc2-f66b-40cf-a0f0-e63dbe6bccc4 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:16.441099 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-f21e3cc2-f66b-40cf-a0f0-e63dbe6bccc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:16.457684 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b1314-31dc-434a-a533-abda1f967260 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:16.469453 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:16.502141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8556e-306b-40e0-903f-1840a8a38198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:16.502141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8556e-306b-40e0-903f-1840a8a38198 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:16.502141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8556e-306b-40e0-903f-1840a8a38198 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:16.502141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8556e-306b-40e0-903f-1840a8a38198 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:16.510047 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8556e-306b-40e0-903f-1840a8a38198 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:16.518006 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8556e-306b-40e0-903f-1840a8a38198 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:16.534078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-f0f8556e-306b-40e0-903f-1840a8a38198 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:16.534078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:16.534078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:16.534078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:16.534078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:16.534078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:16.534078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:16.534078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:16.534078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:16.534078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:16.534078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:16.534078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:16.534078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:16.534078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:16.534078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:16.534078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:16.535083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:16.535083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:16.535083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:16.535083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:16.535083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:16.535083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:16.535083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:16.535083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:16.535083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:16.535083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:16.535083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:16.535083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:16.535083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:16.535083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:16.535083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:16.535083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:16.535083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:16.535973 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:16.535973 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:16.535973 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:16.535973 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 366/727] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:00:16.535973 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8556e-306b-40e0-903f-1840a8a38198 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:16.548020 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 362/728] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:00:16.606127 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:16.616382 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc204c1-9154-4e97-b40b-75432a95bb68 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:16.616664 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc204c1-9154-4e97-b40b-75432a95bb68 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:16.616939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc204c1-9154-4e97-b40b-75432a95bb68 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:16.618517 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 363/729] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:16.639728 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7323ed-2c67-4b1b-bf6c-cb3506a89692 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:16.640000 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7323ed-2c67-4b1b-bf6c-cb3506a89692 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:16.640272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7323ed-2c67-4b1b-bf6c-cb3506a89692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:16.669497 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8556e-306b-40e0-903f-1840a8a38198 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:16.689012 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-2d7323ed-2c67-4b1b-bf6c-cb3506a89692 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:16.690247 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-2d7323ed-2c67-4b1b-bf6c-cb3506a89692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:16.708520 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2991b9c-10ca-43d9-9bb5-dcdd1a1db297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:16.709286 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2991b9c-10ca-43d9-9bb5-dcdd1a1db297 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:16.709671 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2991b9c-10ca-43d9-9bb5-dcdd1a1db297 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:16.710073 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2991b9c-10ca-43d9-9bb5-dcdd1a1db297 None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:16.710826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2991b9c-10ca-43d9-9bb5-dcdd1a1db297 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:16.711247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2991b9c-10ca-43d9-9bb5-dcdd1a1db297 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:16.712719 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2991b9c-10ca-43d9-9bb5-dcdd1a1db297 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:16.726866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 367/730] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:16.732419 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2991b9c-10ca-43d9-9bb5-dcdd1a1db297 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:16.785870 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 364/731] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:00:16.786835 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7323ed-2c67-4b1b-bf6c-cb3506a89692 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:16.843799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2991b9c-10ca-43d9-9bb5-dcdd1a1db297 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:16.878134 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:16.887842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02fda6-5118-417d-91db-47bc6e6d4361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:16.889094 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02fda6-5118-417d-91db-47bc6e6d4361 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:16.889365 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02fda6-5118-417d-91db-47bc6e6d4361 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:16.889639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02fda6-5118-417d-91db-47bc6e6d4361 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:16.905514 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b02fda6-5118-417d-91db-47bc6e6d4361 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:00:16.912388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-1b02fda6-5118-417d-91db-47bc6e6d4361 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:00:16.912388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:16.912388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:16.912388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:16.912388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:16.912388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:16.912388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:16.912388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:16.912388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:16.912388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:16.912388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:16.912388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:16.912388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:00:16.912388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:00:16.912388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:00:16.912388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:00:16.913547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:00:16.913547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:00:16.913547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:00:16.913547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:00:16.913547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:16.913547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:16.913547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:16.913547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:16.913547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:16.913547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:16.913547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:16.913547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:16.913547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:16.913547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:16.913547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:16.913547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:16.914666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:16.914666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:16.914666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:16.914666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:16.914666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:00:16.914666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:00:16.914666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:00:16.914666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:00:16.914666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:00:16.914666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:00:16.914666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:00:16.914666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:16.914666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 368/732] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 20:00:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:16.921297 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02fda6-5118-417d-91db-47bc6e6d4361 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:16.960906 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:16.971276 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7323ed-2c67-4b1b-bf6c-cb3506a89692 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:17.010338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077010f7-7e66-453d-96de-43493062a631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:17.010963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077010f7-7e66-453d-96de-43493062a631 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:17.011290 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077010f7-7e66-453d-96de-43493062a631 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:17.011570 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077010f7-7e66-453d-96de-43493062a631 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:17.020228 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077010f7-7e66-453d-96de-43493062a631 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:17.022042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077010f7-7e66-453d-96de-43493062a631 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:17.038321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-077010f7-7e66-453d-96de-43493062a631 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:00:17.038321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:17.038321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:17.038321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:17.038321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:17.038321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:17.038321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:17.038321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:17.038321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:17.038321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:17.038321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:17.038321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:17.038321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:00:17.038321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:00:17.038321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:00:17.038321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:00:17.039286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:17.039286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:17.039286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:17.039286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:17.039286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:17.039286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:17.039286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:17.039286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:17.039286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:17.039286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:17.039286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:17.039286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:17.039286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:17.039286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:17.039286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:17.039286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:17.039286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:17.039286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:17.040257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:17.040257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:17.040257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:00:17.040257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:00:17.040257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:00:17.040257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:00:17.040257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:00:17.040257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:00:17.040257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:00:17.040257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:17.040257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 365/733] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 20:00:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:17.040851 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077010f7-7e66-453d-96de-43493062a631 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:17.075211 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02fda6-5118-417d-91db-47bc6e6d4361 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:17.117724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512cbe2-795f-4657-b767-6d8d50ca251d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:17.118279 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512cbe2-795f-4657-b767-6d8d50ca251d None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:17.118563 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512cbe2-795f-4657-b767-6d8d50ca251d None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:17.118888 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512cbe2-795f-4657-b767-6d8d50ca251d None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:17.119619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512cbe2-795f-4657-b767-6d8d50ca251d None admin] RBAC: Adding query filter params (name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:17.119972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512cbe2-795f-4657-b767-6d8d50ca251d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:17.121346 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512cbe2-795f-4657-b767-6d8d50ca251d None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:17.136013 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 369/734] 10.222.0.73 () {56 vars in 1103 bytes} [Mon Aug 29 20:00:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:17.137128 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512cbe2-795f-4657-b767-6d8d50ca251d None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:17.156111 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077010f7-7e66-453d-96de-43493062a631 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:17.157131 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:17.189234 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff531ade-e912-4c19-85cb-0f776ec909c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:17.189869 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff531ade-e912-4c19-85cb-0f776ec909c3 None admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:17.190153 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff531ade-e912-4c19-85cb-0f776ec909c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:17.190423 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff531ade-e912-4c19-85cb-0f776ec909c3 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:17.191090 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff531ade-e912-4c19-85cb-0f776ec909c3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:17.192438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff531ade-e912-4c19-85cb-0f776ec909c3 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:17.209399 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 366/735] 10.222.0.73 () {60 vars in 1141 bytes} [Mon Aug 29 20:00:17 2022] POST /identity/v3/services => generated 251 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:00:17.210471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff531ade-e912-4c19-85cb-0f776ec909c3 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:17.228504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:17.254645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512cbe2-795f-4657-b767-6d8d50ca251d None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:17.304829 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a205f0a-3a65-4c7d-8ba2-09e5cf255335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:17.305398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a205f0a-3a65-4c7d-8ba2-09e5cf255335 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:17.305680 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a205f0a-3a65-4c7d-8ba2-09e5cf255335 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:17.305974 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a205f0a-3a65-4c7d-8ba2-09e5cf255335 None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:17.307176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a205f0a-3a65-4c7d-8ba2-09e5cf255335 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:17.307441 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a205f0a-3a65-4c7d-8ba2-09e5cf255335 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:17.308953 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a205f0a-3a65-4c7d-8ba2-09e5cf255335 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:17.330031 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 370/736] 10.222.0.73 () {56 vars in 1135 bytes} [Mon Aug 29 20:00:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:17.331366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f710b-cdd5-441f-8c25-fce4c02831d9 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:17.331650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f710b-cdd5-441f-8c25-fce4c02831d9 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:17.331919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f710b-cdd5-441f-8c25-fce4c02831d9 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:17.333424 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 371/737] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:17.333944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f710b-cdd5-441f-8c25-fce4c02831d9 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:17.338030 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff531ade-e912-4c19-85cb-0f776ec909c3 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:17.381656 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0580f-c10e-45b9-a706-d188454f42ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:17.382561 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0580f-c10e-45b9-a706-d188454f42ad None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:17.383066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0580f-c10e-45b9-a706-d188454f42ad None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:17.383464 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0580f-c10e-45b9-a706-d188454f42ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:17.396764 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0580f-c10e-45b9-a706-d188454f42ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:00:17.402168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-eaa0580f-c10e-45b9-a706-d188454f42ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:00:17.402168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:17.402168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:17.402168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:17.402168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:17.402168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:17.402168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:17.402168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:17.402168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:17.402168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:17.402168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:17.402168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:17.402168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:00:17.402168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:00:17.402168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:00:17.402168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:00:17.403178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:00:17.403178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:00:17.403178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:00:17.403178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:00:17.403178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:17.403178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:17.403178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:17.403178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:17.403178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:17.403178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:17.403178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:17.403178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:17.403178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:17.403178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:17.403178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:17.403178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:17.404062 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:17.404062 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:17.404062 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:17.404062 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:17.404062 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:00:17.404062 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:00:17.404062 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:00:17.404062 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:00:17.404062 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:00:17.404062 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:00:17.404062 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:00:17.404062 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:17.404062 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 367/738] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 20:00:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:17.404745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0580f-c10e-45b9-a706-d188454f42ad None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:17.424737 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:17.458008 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f710b-cdd5-441f-8c25-fce4c02831d9 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:17.485698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:17.496710 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80084d96-0fc4-4a72-8fc9-018518f3d775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:17.497349 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80084d96-0fc4-4a72-8fc9-018518f3d775 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:17.498226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80084d96-0fc4-4a72-8fc9-018518f3d775 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:17.498444 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80084d96-0fc4-4a72-8fc9-018518f3d775 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:17.509814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80084d96-0fc4-4a72-8fc9-018518f3d775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:17.511263 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80084d96-0fc4-4a72-8fc9-018518f3d775 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:17.515594 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0580f-c10e-45b9-a706-d188454f42ad None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:17.529510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-80084d96-0fc4-4a72-8fc9-018518f3d775 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:17.529510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:17.529510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:17.529510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:17.529510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:17.529510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:17.529510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:17.529510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:17.529510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:17.529510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:17.529510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:17.529510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:17.529510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:17.529510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:17.529510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:17.529510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:17.530571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:17.530571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:17.530571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:17.530571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:17.530571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:17.530571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:17.530571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:17.530571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:17.530571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:17.530571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:17.530571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:17.530571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:17.530571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:17.530571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:17.530571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:17.530571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:17.530571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:17.531467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:17.531467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:17.531467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:17.531467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 372/739] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:17.532411 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbc0d8-ea31-43d6-9cf2-32c0961ee775 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:17.532699 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbc0d8-ea31-43d6-9cf2-32c0961ee775 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:17.532972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbc0d8-ea31-43d6-9cf2-32c0961ee775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:17.551323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-a7dbc0d8-ea31-43d6-9cf2-32c0961ee775 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:17.552527 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a7dbc0d8-ea31-43d6-9cf2-32c0961ee775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:17.558231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a349293-2cc9-4da4-a69f-6ae5800b674f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:17.558781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a349293-2cc9-4da4-a69f-6ae5800b674f None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:17.559067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a349293-2cc9-4da4-a69f-6ae5800b674f None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:17.559343 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a349293-2cc9-4da4-a69f-6ae5800b674f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:17.624388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 373/740] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:17 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:00:17.629934 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dbc0d8-ea31-43d6-9cf2-32c0961ee775 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:17.647368 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a349293-2cc9-4da4-a69f-6ae5800b674f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:17.649587 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a349293-2cc9-4da4-a69f-6ae5800b674f None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:17.682609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:17.738642 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:17.745818 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dbc0d8-ea31-43d6-9cf2-32c0961ee775 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:17.770183 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 368/741] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:00:17 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:00:17.770183 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a349293-2cc9-4da4-a69f-6ae5800b674f None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:17.796447 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8342b6-8bd6-47c0-bf3b-f357197e138f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:17.797509 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8342b6-8bd6-47c0-bf3b-f357197e138f None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:17.798078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8342b6-8bd6-47c0-bf3b-f357197e138f None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:17.798559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8342b6-8bd6-47c0-bf3b-f357197e138f None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:17.799437 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8342b6-8bd6-47c0-bf3b-f357197e138f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:17.799955 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8342b6-8bd6-47c0-bf3b-f357197e138f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:17.801870 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8342b6-8bd6-47c0-bf3b-f357197e138f None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:17.812315 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 374/742] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 20:00:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:17.813494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8342b6-8bd6-47c0-bf3b-f357197e138f None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:17.873941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a349293-2cc9-4da4-a69f-6ae5800b674f None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:17.909757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc02788-d1a4-472f-b7e8-e9e6783805f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:17.910380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc02788-d1a4-472f-b7e8-e9e6783805f8 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:17.910637 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc02788-d1a4-472f-b7e8-e9e6783805f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:17.910918 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc02788-d1a4-472f-b7e8-e9e6783805f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:17.911530 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc02788-d1a4-472f-b7e8-e9e6783805f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:17.911825 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc02788-d1a4-472f-b7e8-e9e6783805f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:17.913311 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc02788-d1a4-472f-b7e8-e9e6783805f8 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:17.926067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 369/743] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:17.926834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc02788-d1a4-472f-b7e8-e9e6783805f8 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:17.932658 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8342b6-8bd6-47c0-bf3b-f357197e138f None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:17.951459 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:17.966805 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:17.972557 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aaaaee-8b59-472b-8356-484564df694c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:17.973185 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aaaaee-8b59-472b-8356-484564df694c None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:17.973458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aaaaee-8b59-472b-8356-484564df694c None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:17.973741 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aaaaee-8b59-472b-8356-484564df694c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:17.985860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aaaaee-8b59-472b-8356-484564df694c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:17.987313 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aaaaee-8b59-472b-8356-484564df694c None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:18.002685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-d0aaaaee-8b59-472b-8356-484564df694c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:18.002685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:18.002685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:18.002685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:18.002685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:18.002685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:18.002685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:18.002685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:18.002685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:18.002685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:18.002685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:18.002685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:18.002685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:18.002685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:18.002685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:18.002685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:18.003663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:18.003663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:18.003663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:18.003663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:18.003663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:18.003663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:18.003663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:18.003663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:18.003663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:18.003663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:18.003663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:18.003663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:18.003663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:18.003663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:18.003663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:18.003663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:18.003663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:18.004552 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:18.004552 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:18.004552 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:18.004552 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 375/744] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:18.004552 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aaaaee-8b59-472b-8356-484564df694c None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:18.050512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc02788-d1a4-472f-b7e8-e9e6783805f8 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:18.095413 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272b92f-fc9e-40ff-9990-7915826ed365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:18.096350 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272b92f-fc9e-40ff-9990-7915826ed365 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:18.096765 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272b92f-fc9e-40ff-9990-7915826ed365 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:18.097167 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272b92f-fc9e-40ff-9990-7915826ed365 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:18.110110 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1272b92f-fc9e-40ff-9990-7915826ed365 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:18.111816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1272b92f-fc9e-40ff-9990-7915826ed365 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:18.130292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-1272b92f-fc9e-40ff-9990-7915826ed365 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:18.130292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:18.130292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:18.130292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:18.130292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:18.130292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:18.130292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:18.130292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:18.130292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:18.130292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:18.130292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:18.130292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:18.130292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:18.130292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:18.130292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:18.130292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:18.131384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:18.131384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:18.131384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:18.131384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:18.131384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:18.131384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:18.131384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:18.131384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:18.131384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:18.131384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:18.131384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:18.131384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:18.131384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:18.131384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:18.131384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:18.131384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:18.131384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:18.132268 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:18.132268 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:18.132268 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:18.137865 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 370/745] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:18.137865 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272b92f-fc9e-40ff-9990-7915826ed365 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:18.227029 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aaaaee-8b59-472b-8356-484564df694c None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:18.237934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:18.254696 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272b92f-fc9e-40ff-9990-7915826ed365 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:18.268052 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e243c055-bd48-44cc-9555-0db9d9a827de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:18.268667 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243c055-bd48-44cc-9555-0db9d9a827de None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:18.268946 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243c055-bd48-44cc-9555-0db9d9a827de None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:18.269217 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243c055-bd48-44cc-9555-0db9d9a827de None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:18.269905 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e243c055-bd48-44cc-9555-0db9d9a827de None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:18.271282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e243c055-bd48-44cc-9555-0db9d9a827de None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:18.293959 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 376/746] 10.222.0.73 () {60 vars in 1141 bytes} [Mon Aug 29 20:00:18 2022] POST /identity/v3/services => generated 240 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:00:18.293959 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e243c055-bd48-44cc-9555-0db9d9a827de None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:18.312471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5517710d-4355-4c53-994e-7ce6d8e2adc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:18.313191 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517710d-4355-4c53-994e-7ce6d8e2adc2 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:18.313474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517710d-4355-4c53-994e-7ce6d8e2adc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:18.313952 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517710d-4355-4c53-994e-7ce6d8e2adc2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:18.323228 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5517710d-4355-4c53-994e-7ce6d8e2adc2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:18.324873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5517710d-4355-4c53-994e-7ce6d8e2adc2 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:18.334111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:18.343177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-5517710d-4355-4c53-994e-7ce6d8e2adc2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:00:18.343177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:18.343177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:18.343177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:18.343177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:18.343177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:18.343177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:18.343177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:18.343177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:18.343177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:18.343177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:18.343177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:18.343177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:00:18.343177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:00:18.343177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:00:18.343177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:00:18.344245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:18.344245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:18.344245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:18.344245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:18.344245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:18.344245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:18.344245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:18.344245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:18.344245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:18.344245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:18.344245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:18.344245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:18.344245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:18.344245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:18.344245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:18.344245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:18.344245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:18.344245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:18.345174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:18.345174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:18.345174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:00:18.345174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:00:18.345174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:00:18.345174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:00:18.345174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:00:18.345174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:00:18.345174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:00:18.345174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:18.345174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 371/747] 10.222.0.73 () {56 vars in 1117 bytes} [Mon Aug 29 20:00:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:18.345808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5517710d-4355-4c53-994e-7ce6d8e2adc2 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:18.448003 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e243c055-bd48-44cc-9555-0db9d9a827de None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:18.470741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5517710d-4355-4c53-994e-7ce6d8e2adc2 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:18.491954 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03937b63-4e13-485e-a4cc-605463a12269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:18.492575 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03937b63-4e13-485e-a4cc-605463a12269 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:18.492854 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03937b63-4e13-485e-a4cc-605463a12269 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:18.493124 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03937b63-4e13-485e-a4cc-605463a12269 None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:18.494349 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03937b63-4e13-485e-a4cc-605463a12269 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:18.494653 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03937b63-4e13-485e-a4cc-605463a12269 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:18.496022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03937b63-4e13-485e-a4cc-605463a12269 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:18.513639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 377/748] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:18.513639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03937b63-4e13-485e-a4cc-605463a12269 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:18.516254 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a405a4-e519-45bf-992e-faa61d12f905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:18.516948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a405a4-e519-45bf-992e-faa61d12f905 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:18.517262 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a405a4-e519-45bf-992e-faa61d12f905 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:18.517566 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a405a4-e519-45bf-992e-faa61d12f905 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:18.518261 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a405a4-e519-45bf-992e-faa61d12f905 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:18.518569 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a405a4-e519-45bf-992e-faa61d12f905 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:18.519969 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a405a4-e519-45bf-992e-faa61d12f905 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:18.535825 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 372/749] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:18.536513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a405a4-e519-45bf-992e-faa61d12f905 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:18.539567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:18.620033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:18.635689 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03937b63-4e13-485e-a4cc-605463a12269 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:18.661215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a405a4-e519-45bf-992e-faa61d12f905 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:18.672753 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571749d-a64a-4d3a-bdae-684ca8c7bc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:18.673368 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571749d-a64a-4d3a-bdae-684ca8c7bc45 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:18.673651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571749d-a64a-4d3a-bdae-684ca8c7bc45 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:18.673956 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571749d-a64a-4d3a-bdae-684ca8c7bc45 None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:18.674695 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f571749d-a64a-4d3a-bdae-684ca8c7bc45 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:18.674999 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f571749d-a64a-4d3a-bdae-684ca8c7bc45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:18.676608 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f571749d-a64a-4d3a-bdae-684ca8c7bc45 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:18.696084 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 378/750] 10.222.0.73 () {56 vars in 1149 bytes} [Mon Aug 29 20:00:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:18.697017 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571749d-a64a-4d3a-bdae-684ca8c7bc45 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:18.704583 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821d9d0-e4f4-4be3-980a-92c2d17ad5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:18.706799 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821d9d0-e4f4-4be3-980a-92c2d17ad5b1 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:18.707347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821d9d0-e4f4-4be3-980a-92c2d17ad5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:18.707823 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821d9d0-e4f4-4be3-980a-92c2d17ad5b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:18.716192 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821d9d0-e4f4-4be3-980a-92c2d17ad5b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:18.718385 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821d9d0-e4f4-4be3-980a-92c2d17ad5b1 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:18.731467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-3821d9d0-e4f4-4be3-980a-92c2d17ad5b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:18.731467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:18.731467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:18.731467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:18.731467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:18.731467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:18.731467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:18.731467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:18.731467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:18.731467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:18.731467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:18.731467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:18.731467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:18.731467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:18.731467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:18.733281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:18.733281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:18.733281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:18.733281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:18.733281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:18.733281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:18.733281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:18.733281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:18.733281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:18.733281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:18.733281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:18.733281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:18.733281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:18.733281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:18.733281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:18.733281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:18.733281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:18.734479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:18.734479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:18.734479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:18.734479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:18.734479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:18.734479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:18.734479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:18.734479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:18.736847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 373/751] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:18.738084 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821d9d0-e4f4-4be3-980a-92c2d17ad5b1 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:18.820760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:18.825873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571749d-a64a-4d3a-bdae-684ca8c7bc45 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:18.873323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821d9d0-e4f4-4be3-980a-92c2d17ad5b1 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:18.874565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8a8bf-1556-4efb-88d1-4cd590922d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:18.875211 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8a8bf-1556-4efb-88d1-4cd590922d0b None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:18.875488 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8a8bf-1556-4efb-88d1-4cd590922d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:18.875755 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8a8bf-1556-4efb-88d1-4cd590922d0b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:18.889771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8a8bf-1556-4efb-88d1-4cd590922d0b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:18.891821 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8a8bf-1556-4efb-88d1-4cd590922d0b None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:18.907622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-72c8a8bf-1556-4efb-88d1-4cd590922d0b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:00:18.907622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:18.907622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:18.907622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:18.907622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:18.907622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:18.907622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:18.907622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:18.907622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:18.907622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:18.907622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:18.907622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:18.907622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:00:18.907622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:00:18.907622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:00:18.907622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:00:18.908675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:18.908675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:18.908675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:18.908675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:18.908675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:18.908675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:18.908675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:18.908675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:18.908675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:18.908675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:18.908675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:18.908675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:18.908675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:18.908675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:18.908675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:18.908675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:18.908675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:18.908675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:18.909683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:18.909683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:18.909683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:00:18.909683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:00:18.909683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:00:18.909683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:00:18.909683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:00:18.909683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:00:18.909683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:00:18.909683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:18.909683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 379/752] 10.222.0.73 () {56 vars in 1093 bytes} [Mon Aug 29 20:00:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:18.910340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8a8bf-1556-4efb-88d1-4cd590922d0b None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:18.918670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339183a-cfcf-4dd0-84a4-46b7a5133ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:18.919727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339183a-cfcf-4dd0-84a4-46b7a5133ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:18.920156 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:18.921694 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339183a-cfcf-4dd0-84a4-46b7a5133ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:18.922434 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339183a-cfcf-4dd0-84a4-46b7a5133ce7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:19.006941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7339183a-cfcf-4dd0-84a4-46b7a5133ce7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:19.008398 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7339183a-cfcf-4dd0-84a4-46b7a5133ce7 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:19.030867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8a8bf-1556-4efb-88d1-4cd590922d0b None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:19.068164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7805f67-b055-4412-8078-41332f825e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:19.069166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7805f67-b055-4412-8078-41332f825e90 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:19.069654 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7805f67-b055-4412-8078-41332f825e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:19.070182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7805f67-b055-4412-8078-41332f825e90 None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:19.071084 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7805f67-b055-4412-8078-41332f825e90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:19.071595 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7805f67-b055-4412-8078-41332f825e90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:19.073404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7805f67-b055-4412-8078-41332f825e90 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:19.074130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:19.088680 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 380/753] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:00:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:19.090515 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7805f67-b055-4412-8078-41332f825e90 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:19.129545 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 374/754] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:00:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:00:19.130372 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339183a-cfcf-4dd0-84a4-46b7a5133ce7 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:19.163877 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:19.203668 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7805f67-b055-4412-8078-41332f825e90 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:19.240406 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339183a-cfcf-4dd0-84a4-46b7a5133ce7 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:19.241206 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1947ef2-f334-48df-877b-5eb8692136b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:19.241929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1947ef2-f334-48df-877b-5eb8692136b9 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:19.242362 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1947ef2-f334-48df-877b-5eb8692136b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:19.242803 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1947ef2-f334-48df-877b-5eb8692136b9 None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:19.243625 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1947ef2-f334-48df-877b-5eb8692136b9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:19.243945 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1947ef2-f334-48df-877b-5eb8692136b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:19.245403 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1947ef2-f334-48df-877b-5eb8692136b9 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:19.269515 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 381/755] 10.222.0.73 () {56 vars in 1133 bytes} [Mon Aug 29 20:00:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:19.269515 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1947ef2-f334-48df-877b-5eb8692136b9 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:19.284618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8ad71-80d7-47a1-8da8-3a9197ef1f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:19.284618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8ad71-80d7-47a1-8da8-3a9197ef1f09 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:19.285058 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8ad71-80d7-47a1-8da8-3a9197ef1f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:19.285248 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8ad71-80d7-47a1-8da8-3a9197ef1f09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:19.286055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8ad71-80d7-47a1-8da8-3a9197ef1f09 None admin] RBAC: Adding query filter params (role.id=296ee2c194824485af71618619f82e09, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=b92e30da8f394f3abc19b9e86ef126e5) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:19.286242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8ad71-80d7-47a1-8da8-3a9197ef1f09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:19.287695 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8ad71-80d7-47a1-8da8-3a9197ef1f09 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:19.301420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 375/756] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:00:19 2022] GET /identity/v3/role_assignments?user.id=b92e30da8f394f3abc19b9e86ef126e5&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=296ee2c194824485af71618619f82e09 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:19.301420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2460b3-d18d-40ed-a344-9bbf7d2b9c6f None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:19.301420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2460b3-d18d-40ed-a344-9bbf7d2b9c6f None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:19.301420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2460b3-d18d-40ed-a344-9bbf7d2b9c6f None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:19.301420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 376/757] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:19.301420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2460b3-d18d-40ed-a344-9bbf7d2b9c6f None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:19.339041 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:19.445242 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1947ef2-f334-48df-877b-5eb8692136b9 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:19.463687 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2460b3-d18d-40ed-a344-9bbf7d2b9c6f None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:19.482187 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:19.496451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50232cb-41a9-427c-baae-7a764e24b67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:19.497188 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50232cb-41a9-427c-baae-7a764e24b67c None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:19.498161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50232cb-41a9-427c-baae-7a764e24b67c None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:19.498552 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50232cb-41a9-427c-baae-7a764e24b67c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:19.507878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e7746-3c69-42b1-9c9e-80cf88545df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:19.508644 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e7746-3c69-42b1-9c9e-80cf88545df3 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:19.508989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e7746-3c69-42b1-9c9e-80cf88545df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:19.509331 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e7746-3c69-42b1-9c9e-80cf88545df3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:19.512570 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50232cb-41a9-427c-baae-7a764e24b67c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:19.515119 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50232cb-41a9-427c-baae-7a764e24b67c None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:19.526209 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e7746-3c69-42b1-9c9e-80cf88545df3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:19.527784 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e7746-3c69-42b1-9c9e-80cf88545df3 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:19.535839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-c50232cb-41a9-427c-baae-7a764e24b67c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:19.535839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:19.535839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:19.535839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:19.535839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:19.535839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:19.535839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:19.535839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:19.535839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:19.535839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:19.535839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:19.535839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:19.535839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:19.535839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:19.535839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:19.535839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:19.536849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:19.536849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:19.536849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:19.536849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:19.536849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:19.536849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:19.536849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:19.536849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:19.536849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:19.536849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:19.536849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:19.536849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:19.536849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:19.536849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:19.536849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:19.536849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:19.536849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:19.537813 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:19.537813 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:19.537813 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:19.540049 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 382/758] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:19.541577 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965ca88-9129-49b1-9bca-68c2ae730e04 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:19.542423 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965ca88-9129-49b1-9bca-68c2ae730e04 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:19.542996 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965ca88-9129-49b1-9bca-68c2ae730e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:19.546971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-456e7746-3c69-42b1-9c9e-80cf88545df3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:19.546971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:19.546971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:19.546971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:19.546971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:19.546971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:19.546971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:19.546971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:19.546971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:19.546971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:19.546971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:19.546971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:19.546971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:19.546971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:19.546971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:19.546971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:19.548133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:19.548133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:19.548133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:19.548133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:19.548133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:19.548133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:19.548133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:19.548133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:19.548133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:19.548133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:19.548133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:19.548133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:19.548133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:19.548133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:19.548133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:19.548133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:19.548133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:19.549237 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:19.549237 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:19.549237 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:19.551631 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 377/759] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:19.552758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e7746-3c69-42b1-9c9e-80cf88545df3 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:19.567442 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-b965ca88-9129-49b1-9bca-68c2ae730e04 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:19.569023 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-b965ca88-9129-49b1-9bca-68c2ae730e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:19.653015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:19.656459 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 383/760] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:19.657530 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965ca88-9129-49b1-9bca-68c2ae730e04 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:19.711032 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e7746-3c69-42b1-9c9e-80cf88545df3 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:19.784646 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18cd67-3609-4080-9d2c-39916d66362b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:19.785995 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18cd67-3609-4080-9d2c-39916d66362b None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:19.787082 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18cd67-3609-4080-9d2c-39916d66362b None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:19.787651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18cd67-3609-4080-9d2c-39916d66362b None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:19.789551 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c18cd67-3609-4080-9d2c-39916d66362b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:19.790200 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c18cd67-3609-4080-9d2c-39916d66362b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:19.792460 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c18cd67-3609-4080-9d2c-39916d66362b None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:19.808206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 378/761] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:19.814161 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18cd67-3609-4080-9d2c-39916d66362b None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:19.890204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965ca88-9129-49b1-9bca-68c2ae730e04 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:19.930128 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:19.980415 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77013558-488d-432b-94d2-3c4257c03728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:19.981437 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77013558-488d-432b-94d2-3c4257c03728 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:19.981739 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77013558-488d-432b-94d2-3c4257c03728 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:19.982529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77013558-488d-432b-94d2-3c4257c03728 None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:19.983201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77013558-488d-432b-94d2-3c4257c03728 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:19.983505 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77013558-488d-432b-94d2-3c4257c03728 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:19.984902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77013558-488d-432b-94d2-3c4257c03728 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:20.011479 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 384/762] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:20.013863 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77013558-488d-432b-94d2-3c4257c03728 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:20.032972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18cd67-3609-4080-9d2c-39916d66362b None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:20.078472 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2d65e-805e-4288-a911-9c93bb17aeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:20.079158 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2d65e-805e-4288-a911-9c93bb17aeca None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:20.079435 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2d65e-805e-4288-a911-9c93bb17aeca None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:20.079713 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2d65e-805e-4288-a911-9c93bb17aeca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:20.087831 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2d65e-805e-4288-a911-9c93bb17aeca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:20.089289 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2d65e-805e-4288-a911-9c93bb17aeca None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:20.091233 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:20.114152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-8aa2d65e-805e-4288-a911-9c93bb17aeca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:20.114152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:20.114152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:20.114152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:20.114152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:20.114152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:20.114152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:20.114152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:20.114152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:20.114152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:20.114152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:20.114152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:20.114152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:20.114152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:20.114152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:20.115822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:20.115822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:20.115822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:20.115822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:20.115822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:20.115822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:20.115822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:20.115822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:20.115822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:20.115822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:20.115822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:20.115822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:20.115822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:20.115822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:20.115822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:20.115822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:20.115822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:20.116712 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:20.116712 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:20.116712 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:20.116712 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:20.116712 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:20.116712 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:20.116712 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:20.116712 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:20.116712 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 379/763] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:00:20.116712 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2d65e-805e-4288-a911-9c93bb17aeca None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:20.152584 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77013558-488d-432b-94d2-3c4257c03728 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:20.199460 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3325f22-8eaf-4768-bc69-a1fe5e8c7246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:20.200169 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3325f22-8eaf-4768-bc69-a1fe5e8c7246 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:20.200462 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3325f22-8eaf-4768-bc69-a1fe5e8c7246 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:20.205732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3325f22-8eaf-4768-bc69-a1fe5e8c7246 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:20.205732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3325f22-8eaf-4768-bc69-a1fe5e8c7246 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:20.205732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3325f22-8eaf-4768-bc69-a1fe5e8c7246 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:20.217370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-f3325f22-8eaf-4768-bc69-a1fe5e8c7246 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:00:20.217370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:20.217370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:00:20.217370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:00:20.217370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:00:20.217370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:00:20.217370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:00:20.217370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:20.217370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:00:20.217370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:20.217370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:20.217370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:20.217370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:20.217370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:20.217370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:20.217370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:20.217370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:20.218864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:20.218864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:20.218864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:20.218864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:20.218864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:20.218864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:20.218864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:00:20.218864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:00:20.218864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:00:20.218864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:00:20.218864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:20.218864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:20.218864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:20.218864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:20.218864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:20.218864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:20.219977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:20.219977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:20.219977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:20.219977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:20.219977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:20.219977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:20.219977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:20.219977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:20.219977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:20.219977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:20.219977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:00:20.219977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:20.219977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:00:20.219977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:20.219977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 385/764] 10.222.0.73 () {56 vars in 1129 bytes} [Mon Aug 29 20:00:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:20.221829 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3325f22-8eaf-4768-bc69-a1fe5e8c7246 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:20.247338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2d65e-805e-4288-a911-9c93bb17aeca None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:20.268996 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:20.290872 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f662596-433f-4a99-96ae-239fb2b27ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:20.291609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f662596-433f-4a99-96ae-239fb2b27ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:20.291950 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f662596-433f-4a99-96ae-239fb2b27ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:20.292229 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f662596-433f-4a99-96ae-239fb2b27ed0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:20.300275 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f662596-433f-4a99-96ae-239fb2b27ed0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:20.301643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f662596-433f-4a99-96ae-239fb2b27ed0 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:20.314443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-4f662596-433f-4a99-96ae-239fb2b27ed0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:20.314443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:20.314443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:20.314443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:20.314443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:20.314443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:20.314443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:20.314443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:20.314443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:20.314443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:20.314443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:20.314443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:20.314443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:20.314443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:20.314443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:20.315348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:20.315348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:20.315348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:20.315348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:20.315348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:20.315348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:20.315348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:20.315348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:20.315348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:20.315348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:20.315348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:20.315348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:20.315348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:20.315348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:20.315348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:20.315348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:20.315348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:20.316316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:20.316316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:20.316316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:20.316316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:20.316316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:20.316316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:20.316316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:20.316316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:20.320526 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 380/765] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:20.321508 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f662596-433f-4a99-96ae-239fb2b27ed0 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:20.374994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3325f22-8eaf-4768-bc69-a1fe5e8c7246 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:20.391650 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:20.423631 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1690d7a-9480-47c1-8036-a7a0f29bc1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:20.424246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1690d7a-9480-47c1-8036-a7a0f29bc1ef None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:20.424534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1690d7a-9480-47c1-8036-a7a0f29bc1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:20.424890 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1690d7a-9480-47c1-8036-a7a0f29bc1ef None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:20.425485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1690d7a-9480-47c1-8036-a7a0f29bc1ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:20.425834 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1690d7a-9480-47c1-8036-a7a0f29bc1ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:20.427502 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1690d7a-9480-47c1-8036-a7a0f29bc1ef None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:20.441128 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 386/766] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:00:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:20.442236 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1690d7a-9480-47c1-8036-a7a0f29bc1ef None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:20.447487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f662596-433f-4a99-96ae-239fb2b27ed0 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:20.493235 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389fe83e-84c4-4768-99b4-9f570a7238ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:20.493993 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fe83e-84c4-4768-99b4-9f570a7238ee None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:20.494276 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fe83e-84c4-4768-99b4-9f570a7238ee None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:20.494557 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fe83e-84c4-4768-99b4-9f570a7238ee None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:20.495188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389fe83e-84c4-4768-99b4-9f570a7238ee None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:20.495503 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389fe83e-84c4-4768-99b4-9f570a7238ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:20.496876 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389fe83e-84c4-4768-99b4-9f570a7238ee None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:20.506363 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 381/767] 10.222.0.73 () {56 vars in 1124 bytes} [Mon Aug 29 20:00:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:20.506363 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389fe83e-84c4-4768-99b4-9f570a7238ee None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:20.549553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:20.561834 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1690d7a-9480-47c1-8036-a7a0f29bc1ef None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:20.601594 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e090e-6b50-4ea8-a2a5-3a0d8216beb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:20.602175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e090e-6b50-4ea8-a2a5-3a0d8216beb3 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:20.602452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e090e-6b50-4ea8-a2a5-3a0d8216beb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:20.602723 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e090e-6b50-4ea8-a2a5-3a0d8216beb3 None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:20.603417 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e090e-6b50-4ea8-a2a5-3a0d8216beb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:20.603724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e090e-6b50-4ea8-a2a5-3a0d8216beb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:20.605424 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e090e-6b50-4ea8-a2a5-3a0d8216beb3 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:20.620291 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 387/768] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:00:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:20.622008 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048b79c-32c2-4cd2-9dcc-08b1b41b50c5 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:20.622429 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048b79c-32c2-4cd2-9dcc-08b1b41b50c5 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:20.623143 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389fe83e-84c4-4768-99b4-9f570a7238ee None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:20.623821 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048b79c-32c2-4cd2-9dcc-08b1b41b50c5 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:20.626912 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 388/769] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:20.627936 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a048b79c-32c2-4cd2-9dcc-08b1b41b50c5 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:20.649011 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:20.661959 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c9e90-effa-4c9d-8990-3c7dab23fd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:20.662575 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c9e90-effa-4c9d-8990-3c7dab23fd7f None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:20.662858 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c9e90-effa-4c9d-8990-3c7dab23fd7f None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:20.663131 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c9e90-effa-4c9d-8990-3c7dab23fd7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:20.663850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c9e90-effa-4c9d-8990-3c7dab23fd7f None admin] RBAC: Adding query filter params (role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=e0c76478b6694782b896cdf9cda32d9c) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:20.664162 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c9e90-effa-4c9d-8990-3c7dab23fd7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:20.665610 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c9e90-effa-4c9d-8990-3c7dab23fd7f None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:20.672378 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 382/770] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:00:20 2022] GET /identity/v3/role_assignments?user.id=e0c76478b6694782b896cdf9cda32d9c&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:20.672974 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c9e90-effa-4c9d-8990-3c7dab23fd7f None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:20.763993 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a048b79c-32c2-4cd2-9dcc-08b1b41b50c5 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:20.800531 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c9e90-effa-4c9d-8990-3c7dab23fd7f None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:20.802087 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3697c-9efa-4513-b08c-ebe670fe08f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:20.802825 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3697c-9efa-4513-b08c-ebe670fe08f4 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:20.803205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3697c-9efa-4513-b08c-ebe670fe08f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:20.803586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3697c-9efa-4513-b08c-ebe670fe08f4 None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:20.805173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b3697c-9efa-4513-b08c-ebe670fe08f4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:20.805173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b3697c-9efa-4513-b08c-ebe670fe08f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:20.806243 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b3697c-9efa-4513-b08c-ebe670fe08f4 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:20.812550 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 389/771] 10.222.0.73 () {56 vars in 1124 bytes} [Mon Aug 29 20:00:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:20.814009 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05489d22-cad8-49ed-b8b8-305e4ec40e8a None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:20.814413 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05489d22-cad8-49ed-b8b8-305e4ec40e8a None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:20.814838 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05489d22-cad8-49ed-b8b8-305e4ec40e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:20.838539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-05489d22-cad8-49ed-b8b8-305e4ec40e8a None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:20.839268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-05489d22-cad8-49ed-b8b8-305e4ec40e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:20.847891 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ba083-35a6-4732-8e20-1a59851271ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:20.849430 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ba083-35a6-4732-8e20-1a59851271ef None admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:20.849896 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ba083-35a6-4732-8e20-1a59851271ef None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:20.850340 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ba083-35a6-4732-8e20-1a59851271ef None admin] PATH_INFO: `/v3/projects/2f5ec615d98548159487c97cead336a0/users/95cf77a92a174ff8b0503baf5c7e63be/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:20.852001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:20.855766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64ba083-35a6-4732-8e20-1a59851271ef None admin] RBAC: Authorizing `identity:create_grant(project_id=2f5ec615d98548159487c97cead336a0, user_id=95cf77a92a174ff8b0503baf5c7e63be, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:20.858433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64ba083-35a6-4732-8e20-1a59851271ef None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:20.879830 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 383/772] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 20:00:20 2022] PUT /identity/v3/projects/2f5ec615d98548159487c97cead336a0/users/95cf77a92a174ff8b0503baf5c7e63be/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:00:20.881003 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ba083-35a6-4732-8e20-1a59851271ef None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:20.926707 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 390/773] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:20.947070 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:21.018706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ba083-35a6-4732-8e20-1a59851271ef None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:21.059569 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ccb5a-52be-4ded-b41d-56129062b13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:21.061037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ccb5a-52be-4ded-b41d-56129062b13b None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:21.061512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ccb5a-52be-4ded-b41d-56129062b13b None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:21.062082 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ccb5a-52be-4ded-b41d-56129062b13b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:21.063002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ccb5a-52be-4ded-b41d-56129062b13b None admin] RBAC: Adding query filter params (interface=public, service_id=22ecd51f144f4b7cbe5c2da2d1d476ef, region_id=RegionOne) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:21.063483 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ccb5a-52be-4ded-b41d-56129062b13b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:21.065093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ccb5a-52be-4ded-b41d-56129062b13b None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:21.074621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 384/774] 10.222.0.73 () {56 vars in 1251 bytes} [Mon Aug 29 20:00:20 2022] GET /identity/v3/endpoints?service_id=22ecd51f144f4b7cbe5c2da2d1d476ef&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:21.079887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05489d22-cad8-49ed-b8b8-305e4ec40e8a None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:21.201788 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05489d22-cad8-49ed-b8b8-305e4ec40e8a None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:21.240938 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623eca36-aa0d-4814-aa80-bf1e8f508d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:21.241840 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623eca36-aa0d-4814-aa80-bf1e8f508d5b None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:21.242270 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623eca36-aa0d-4814-aa80-bf1e8f508d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:21.242719 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623eca36-aa0d-4814-aa80-bf1e8f508d5b None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:21.243471 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623eca36-aa0d-4814-aa80-bf1e8f508d5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:21.245016 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623eca36-aa0d-4814-aa80-bf1e8f508d5b None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:21.252057 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 391/775] 10.222.0.73 () {56 vars in 1087 bytes} [Mon Aug 29 20:00:21 2022] GET /identity/v3/services => generated 772 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:21.284356 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ccb5a-52be-4ded-b41d-56129062b13b None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:21.401835 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ccb5a-52be-4ded-b41d-56129062b13b None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:21.414093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:21.442038 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01d9e0-15dd-44df-9821-c7543034d557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:21.442857 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01d9e0-15dd-44df-9821-c7543034d557 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:21.443145 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01d9e0-15dd-44df-9821-c7543034d557 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:21.443419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01d9e0-15dd-44df-9821-c7543034d557 None admin] PATH_INFO: `/v3/services/network` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:21.444018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01d9e0-15dd-44df-9821-c7543034d557 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:21.445427 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01d9e0-15dd-44df-9821-c7543034d557 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:21.457792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-8a01d9e0-15dd-44df-9821-c7543034d557 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:00:21.457792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:21.457792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:00:21.457792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:00:21.457792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:00:21.457792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:00:21.457792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:00:21.457792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:21.457792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:00:21.457792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:21.457792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:21.457792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:21.457792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:21.457792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:21.457792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:21.457792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:21.457792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:21.458840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:21.458840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:21.458840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:21.458840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:21.458840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:21.458840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:21.458840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:00:21.458840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:00:21.458840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:00:21.458840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:00:21.458840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:21.458840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:21.458840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:21.458840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:21.458840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:21.458840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:21.459676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:21.459676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:21.459676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:21.459676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:21.459676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:21.459676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:21.459676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:21.459676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:21.459676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:21.459676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:21.459676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:00:21.459676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:21.459676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:00:21.459676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:21.459676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 385/776] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:21 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:21.465107 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623eca36-aa0d-4814-aa80-bf1e8f508d5b None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:21.482886 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b206fc-ae41-459e-bfd7-72e5b60983aa None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:21.483203 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b206fc-ae41-459e-bfd7-72e5b60983aa None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:21.483429 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b206fc-ae41-459e-bfd7-72e5b60983aa None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:21.484935 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 386/777] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:00:21.486162 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:21.491088 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58585e-5588-49c4-b512-954197a86641 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:21.491389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58585e-5588-49c4-b512-954197a86641 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:21.491645 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58585e-5588-49c4-b512-954197a86641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:21.512490 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-7d58585e-5588-49c4-b512-954197a86641 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:21.513836 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-7d58585e-5588-49c4-b512-954197a86641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:21.606495 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 387/778] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:00:21.624730 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623eca36-aa0d-4814-aa80-bf1e8f508d5b None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:21.669550 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745d59c-72cb-4852-9d05-943f54d9214d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:21.670709 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745d59c-72cb-4852-9d05-943f54d9214d None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:21.670988 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745d59c-72cb-4852-9d05-943f54d9214d None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:21.671263 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745d59c-72cb-4852-9d05-943f54d9214d None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:21.671896 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745d59c-72cb-4852-9d05-943f54d9214d None admin] RBAC: Adding query filter params (name=network) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:21.672195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745d59c-72cb-4852-9d05-943f54d9214d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:21.673566 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745d59c-72cb-4852-9d05-943f54d9214d None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:21.678792 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 392/779] 10.222.0.73 () {56 vars in 1112 bytes} [Mon Aug 29 20:00:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:21.684273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58585e-5588-49c4-b512-954197a86641 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:21.827718 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58585e-5588-49c4-b512-954197a86641 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:21.830340 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:22.005818 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745d59c-72cb-4852-9d05-943f54d9214d None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:22.006107 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2da812-3d63-4460-98a3-77cc40674456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:22.006107 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2da812-3d63-4460-98a3-77cc40674456 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:22.006107 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2da812-3d63-4460-98a3-77cc40674456 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:22.006107 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2da812-3d63-4460-98a3-77cc40674456 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:22.006718 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2da812-3d63-4460-98a3-77cc40674456 None admin] RBAC: Adding query filter params (type=network) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:22.007016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2da812-3d63-4460-98a3-77cc40674456 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:22.008388 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2da812-3d63-4460-98a3-77cc40674456 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:22.029806 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 388/780] 10.222.0.73 () {56 vars in 1112 bytes} [Mon Aug 29 20:00:21 2022] GET /identity/v3/services?type=network => generated 350 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:22.036449 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2da812-3d63-4460-98a3-77cc40674456 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:22.114913 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:22.156149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745d59c-72cb-4852-9d05-943f54d9214d None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:22.198056 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a72fb64-3a94-4430-a09d-1a10268880f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:22.202097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72fb64-3a94-4430-a09d-1a10268880f0 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:22.203685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72fb64-3a94-4430-a09d-1a10268880f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:22.204990 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72fb64-3a94-4430-a09d-1a10268880f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:22.231768 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a72fb64-3a94-4430-a09d-1a10268880f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:00:22.238258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-0a72fb64-3a94-4430-a09d-1a10268880f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:00:22.238258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:22.238258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:22.238258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:22.238258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:22.238258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:22.238258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:22.238258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:22.238258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:22.238258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:22.238258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:22.238258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:22.238258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:00:22.238258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:00:22.238258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:00:22.238258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:00:22.239210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:00:22.239210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:00:22.239210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:00:22.239210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:00:22.239210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:22.239210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:22.239210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:22.239210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:22.239210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:22.239210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:22.239210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:22.239210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:22.239210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:22.239210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:22.239210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:22.239210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:22.240060 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:22.240060 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:22.240060 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:22.240060 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:22.240060 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:00:22.240060 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:00:22.240060 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:00:22.240060 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:00:22.240060 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:00:22.240060 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:00:22.240060 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:00:22.240060 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:22.257353 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 393/781] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 20:00:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:22.258614 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a72fb64-3a94-4430-a09d-1a10268880f0 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:22.311712 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2da812-3d63-4460-98a3-77cc40674456 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:22.380776 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d94bf-294c-4456-9e32-55b2884e51d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:22.383132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d94bf-294c-4456-9e32-55b2884e51d4 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:22.383589 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d94bf-294c-4456-9e32-55b2884e51d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:22.384285 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d94bf-294c-4456-9e32-55b2884e51d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:22.386392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d94bf-294c-4456-9e32-55b2884e51d4 None admin] RBAC: Adding query filter params (interface=public, service_id=cb6d94abdd5047f48e9a40845bccd62b, region_id=RegionOne) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:22.386922 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d94bf-294c-4456-9e32-55b2884e51d4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:22.389594 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d94bf-294c-4456-9e32-55b2884e51d4 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:22.409475 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 389/782] 10.222.0.73 () {56 vars in 1251 bytes} [Mon Aug 29 20:00:22 2022] GET /identity/v3/endpoints?service_id=cb6d94abdd5047f48e9a40845bccd62b&interface=public®ion_id=RegionOne => generated 194 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:22.410689 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d94bf-294c-4456-9e32-55b2884e51d4 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:22.481637 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:22.485700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a72fb64-3a94-4430-a09d-1a10268880f0 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:22.555997 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:22.566125 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c109b-2f1e-4d56-8839-2dd73fe59355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:22.571640 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c109b-2f1e-4d56-8839-2dd73fe59355 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:22.572031 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c109b-2f1e-4d56-8839-2dd73fe59355 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:22.572377 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c109b-2f1e-4d56-8839-2dd73fe59355 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:22.573176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c109b-2f1e-4d56-8839-2dd73fe59355 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:22.573551 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c109b-2f1e-4d56-8839-2dd73fe59355 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:22.575066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c109b-2f1e-4d56-8839-2dd73fe59355 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:22.598664 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 394/783] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 20:00:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:22.599676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c109b-2f1e-4d56-8839-2dd73fe59355 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:22.636348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d94bf-294c-4456-9e32-55b2884e51d4 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:22.710855 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeee97b-2d7b-4019-a65e-54b06e33d805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:22.713312 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeee97b-2d7b-4019-a65e-54b06e33d805 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:22.713674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeee97b-2d7b-4019-a65e-54b06e33d805 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:22.714074 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeee97b-2d7b-4019-a65e-54b06e33d805 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:22.716967 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeee97b-2d7b-4019-a65e-54b06e33d805 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:22.719723 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeee97b-2d7b-4019-a65e-54b06e33d805 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:22.735775 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 390/784] 10.222.0.73 () {56 vars in 1087 bytes} [Mon Aug 29 20:00:22 2022] GET /identity/v3/services => generated 772 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:22.836633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c109b-2f1e-4d56-8839-2dd73fe59355 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:22.916085 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e09edb-d8d7-4f6f-aa43-ed2ce9a6c5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:22.916756 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e09edb-d8d7-4f6f-aa43-ed2ce9a6c5d3 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:22.917036 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e09edb-d8d7-4f6f-aa43-ed2ce9a6c5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:22.917307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e09edb-d8d7-4f6f-aa43-ed2ce9a6c5d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:22.944803 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e09edb-d8d7-4f6f-aa43-ed2ce9a6c5d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:22.945763 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e09edb-d8d7-4f6f-aa43-ed2ce9a6c5d3 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:22.971176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-a9e09edb-d8d7-4f6f-aa43-ed2ce9a6c5d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:22.971176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:22.971176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:22.971176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:22.971176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:22.971176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:22.971176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:22.971176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:22.971176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:22.971176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:22.971176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:22.971176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:22.971176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:22.971176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:22.971176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:22.971176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:22.972420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:22.972420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:22.972420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:22.972420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:22.972420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:22.972420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:22.972420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:22.972420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:22.972420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:22.972420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:22.972420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:22.972420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:22.972420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:22.972420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:22.972420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:22.972420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:22.972420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:22.973512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:22.973512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:22.973512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:22.973512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 395/785] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:22.979333 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeee97b-2d7b-4019-a65e-54b06e33d805 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:23.096996 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeee97b-2d7b-4019-a65e-54b06e33d805 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:23.102823 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:23.133838 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a1cd3-e229-4e5b-ae07-4ba3c745164e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:23.134443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a1cd3-e229-4e5b-ae07-4ba3c745164e None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:23.134716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a1cd3-e229-4e5b-ae07-4ba3c745164e None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:23.134981 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a1cd3-e229-4e5b-ae07-4ba3c745164e None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:23.135614 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a1cd3-e229-4e5b-ae07-4ba3c745164e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:23.136211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a1cd3-e229-4e5b-ae07-4ba3c745164e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:23.137562 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a1cd3-e229-4e5b-ae07-4ba3c745164e None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:23.147700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395861a-2d8c-4a4a-9b7e-4549dc9789ca None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:23.148042 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395861a-2d8c-4a4a-9b7e-4549dc9789ca None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:23.148260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395861a-2d8c-4a4a-9b7e-4549dc9789ca None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:23.149600 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 396/786] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:23.150843 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 391/787] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:23.155309 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d96f7-8092-4a38-b56a-8a36bdf21b80 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:23.155610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d96f7-8092-4a38-b56a-8a36bdf21b80 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:23.155999 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d96f7-8092-4a38-b56a-8a36bdf21b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:23.156921 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a1cd3-e229-4e5b-ae07-4ba3c745164e None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:23.190863 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-c07d96f7-8092-4a38-b56a-8a36bdf21b80 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:23.192444 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c07d96f7-8092-4a38-b56a-8a36bdf21b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:23.282152 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:23.335547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 397/788] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:23.346485 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a1cd3-e229-4e5b-ae07-4ba3c745164e None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:23.388105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e5a32-c990-4950-89ee-e98dc6033cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:23.389217 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e5a32-c990-4950-89ee-e98dc6033cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:23.389495 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e5a32-c990-4950-89ee-e98dc6033cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:23.389794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e5a32-c990-4950-89ee-e98dc6033cb7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:23.396135 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e8ee2-9ac0-4145-90ec-4f519986bb83 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:23.396418 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e8ee2-9ac0-4145-90ec-4f519986bb83 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:23.396696 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e8ee2-9ac0-4145-90ec-4f519986bb83 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:23.398392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 398/789] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:23.402808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e5a32-c990-4950-89ee-e98dc6033cb7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:23.403560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49346266-984e-4241-8186-7af0267c78c0 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:23.404155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49346266-984e-4241-8186-7af0267c78c0 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:23.404265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e5a32-c990-4950-89ee-e98dc6033cb7 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:23.404744 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49346266-984e-4241-8186-7af0267c78c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:23.416346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-055e5a32-c990-4950-89ee-e98dc6033cb7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:00:23.416346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:23.416346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:23.416346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:23.416346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:23.416346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:23.416346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:23.416346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:23.416346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:23.416346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:23.416346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:23.416346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:23.416346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:00:23.416346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:00:23.416346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:00:23.416346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:00:23.417305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:23.417305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:23.417305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:23.417305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:23.417305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:23.417305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:23.417305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:23.417305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:23.417305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:23.417305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:23.417305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:23.417305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:23.417305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:23.417305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:23.417305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:23.417305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:23.417305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:23.417305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:23.418659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:23.418659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:23.418659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:00:23.418659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:00:23.418659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:00:23.418659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:00:23.418659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:00:23.418659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:00:23.418659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:00:23.418659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:23.418659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 392/790] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 20:00:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:00:23.426655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e5a32-c990-4950-89ee-e98dc6033cb7 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:23.445742 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-49346266-984e-4241-8186-7af0267c78c0 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:23.445742 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-49346266-984e-4241-8186-7af0267c78c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:23.468180 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:23.585081 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 399/791] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:23.619155 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e5a32-c990-4950-89ee-e98dc6033cb7 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:23.663915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49346266-984e-4241-8186-7af0267c78c0 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:23.664849 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ded35-5ba2-4817-a964-a69c9187c2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:23.665679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ded35-5ba2-4817-a964-a69c9187c2b9 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:23.666209 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ded35-5ba2-4817-a964-a69c9187c2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:23.666677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ded35-5ba2-4817-a964-a69c9187c2b9 None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:23.667608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ded35-5ba2-4817-a964-a69c9187c2b9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:23.668104 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ded35-5ba2-4817-a964-a69c9187c2b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:23.670292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ded35-5ba2-4817-a964-a69c9187c2b9 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:23.704197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 393/792] 10.222.0.73 () {56 vars in 1135 bytes} [Mon Aug 29 20:00:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:23.711596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ded35-5ba2-4817-a964-a69c9187c2b9 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:23.840943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49346266-984e-4241-8186-7af0267c78c0 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:23.843595 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:23.858413 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:23.876627 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ded35-5ba2-4817-a964-a69c9187c2b9 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:23.910744 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f08e98-d457-402c-b450-34376981f9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:23.911345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f08e98-d457-402c-b450-34376981f9f8 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:23.911627 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f08e98-d457-402c-b450-34376981f9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:23.911902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f08e98-d457-402c-b450-34376981f9f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:23.915896 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc7b76-31f5-48b4-ba37-a2e6db3de271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:23.916775 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc7b76-31f5-48b4-ba37-a2e6db3de271 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:23.917246 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc7b76-31f5-48b4-ba37-a2e6db3de271 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:23.917741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc7b76-31f5-48b4-ba37-a2e6db3de271 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:23.923948 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f08e98-d457-402c-b450-34376981f9f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:00:23.929166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-d1f08e98-d457-402c-b450-34376981f9f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:00:23.929166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:23.929166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:23.929166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:23.929166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:23.929166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:23.929166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:23.929166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:23.929166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:23.929166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:23.929166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:23.929166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:23.929166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:00:23.929166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:00:23.929166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:00:23.929166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:00:23.930149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:00:23.930149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:00:23.930149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:00:23.930149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:00:23.930149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:23.930149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:23.930149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:23.930149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:23.930149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:23.930149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:23.930149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:23.930149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:23.930149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:23.930149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:23.930149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:23.930149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:23.930994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:23.930994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:23.930994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:23.930994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:23.930994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:00:23.930994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:00:23.930994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:00:23.930994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:00:23.930994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:00:23.930994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:00:23.930994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:00:23.930994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:23.930994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 400/793] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 20:00:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:00:23.931682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f08e98-d457-402c-b450-34376981f9f8 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:23.933368 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc7b76-31f5-48b4-ba37-a2e6db3de271 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:23.934874 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc7b76-31f5-48b4-ba37-a2e6db3de271 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:23.950991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-9bdc7b76-31f5-48b4-ba37-a2e6db3de271 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:23.950991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:23.950991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:23.950991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:23.950991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:23.950991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:23.950991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:23.950991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:23.950991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:23.950991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:23.950991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:23.950991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:23.950991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:23.950991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:23.950991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:23.950991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:23.952040 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:23.952040 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:23.952040 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:23.952040 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:23.952040 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:23.952040 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:23.952040 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:23.952040 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:23.952040 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:23.952040 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:23.952040 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:23.952040 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:23.952040 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:23.952040 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:23.952040 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:23.952040 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:23.952040 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:23.952917 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:23.952917 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:23.952917 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:23.952917 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 394/794] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:00:23.953556 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67a9c2-25b9-4145-a5e1-d0ecf248bd20 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:23.953791 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67a9c2-25b9-4145-a5e1-d0ecf248bd20 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:23.954068 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67a9c2-25b9-4145-a5e1-d0ecf248bd20 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:23.957608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 395/795] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:23.957608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67a9c2-25b9-4145-a5e1-d0ecf248bd20 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:24.051770 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f08e98-d457-402c-b450-34376981f9f8 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:24.072968 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67a9c2-25b9-4145-a5e1-d0ecf248bd20 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:24.091361 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608afc10-17ab-43ba-997b-396851fc1eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:24.092547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608afc10-17ab-43ba-997b-396851fc1eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:24.092885 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608afc10-17ab-43ba-997b-396851fc1eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:24.093090 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608afc10-17ab-43ba-997b-396851fc1eb3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:24.106272 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608afc10-17ab-43ba-997b-396851fc1eb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:00:24.108951 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:24.112317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-608afc10-17ab-43ba-997b-396851fc1eb3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:00:24.112317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:24.112317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:24.112317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:24.112317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:24.112317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:24.112317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:24.112317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:24.112317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:24.112317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:24.112317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:24.112317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:24.112317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:00:24.112317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:00:24.112317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:00:24.112317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:00:24.113254 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:00:24.113254 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:00:24.113254 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:00:24.113254 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:00:24.113254 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:24.113254 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:24.113254 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:24.113254 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:24.113254 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:24.113254 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:24.113254 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:24.113254 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:24.113254 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:24.113254 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:24.113254 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:24.113254 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:24.114120 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:24.114120 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:24.114120 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:24.114120 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:24.114120 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:00:24.114120 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:00:24.114120 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:00:24.114120 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:00:24.114120 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:00:24.114120 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:00:24.114120 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:00:24.114120 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:24.114120 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 401/796] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 20:00:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:24.114810 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608afc10-17ab-43ba-997b-396851fc1eb3 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:24.123517 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0d2d7-df9f-4113-8b64-287aea748fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:24.124181 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0d2d7-df9f-4113-8b64-287aea748fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:24.124454 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0d2d7-df9f-4113-8b64-287aea748fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:24.124984 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0d2d7-df9f-4113-8b64-287aea748fe3 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:24.125675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0d2d7-df9f-4113-8b64-287aea748fe3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:24.126051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0d2d7-df9f-4113-8b64-287aea748fe3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:24.127430 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0d2d7-df9f-4113-8b64-287aea748fe3 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:24.136337 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 396/797] 10.222.0.73 () {56 vars in 1103 bytes} [Mon Aug 29 20:00:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:24.142922 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7433e42-8229-4dfa-8098-128a2e3d624c None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:24.143190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7433e42-8229-4dfa-8098-128a2e3d624c None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:24.143416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7433e42-8229-4dfa-8098-128a2e3d624c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:24.149931 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:24.162962 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-c7433e42-8229-4dfa-8098-128a2e3d624c None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:24.164183 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c7433e42-8229-4dfa-8098-128a2e3d624c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:24.232736 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608afc10-17ab-43ba-997b-396851fc1eb3 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:24.235332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 397/798] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:24.236342 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7433e42-8229-4dfa-8098-128a2e3d624c None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:24.278969 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37aed4-3c17-4948-a46f-11ee1f86df60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:24.279942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37aed4-3c17-4948-a46f-11ee1f86df60 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:24.280334 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37aed4-3c17-4948-a46f-11ee1f86df60 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:24.280701 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37aed4-3c17-4948-a46f-11ee1f86df60 None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:24.281440 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37aed4-3c17-4948-a46f-11ee1f86df60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:24.281865 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37aed4-3c17-4948-a46f-11ee1f86df60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:24.283328 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37aed4-3c17-4948-a46f-11ee1f86df60 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:24.296477 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 402/799] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:24.297510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37aed4-3c17-4948-a46f-11ee1f86df60 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:24.347100 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7433e42-8229-4dfa-8098-128a2e3d624c None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:24.366469 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:24.388654 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e02c18f-58bc-44c0-a464-24673b5b4abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:24.389576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02c18f-58bc-44c0-a464-24673b5b4abe None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:24.390100 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02c18f-58bc-44c0-a464-24673b5b4abe None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:24.390554 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02c18f-58bc-44c0-a464-24673b5b4abe None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:24.391403 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e02c18f-58bc-44c0-a464-24673b5b4abe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:24.391886 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e02c18f-58bc-44c0-a464-24673b5b4abe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:24.393513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e02c18f-58bc-44c0-a464-24673b5b4abe None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:24.401034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:24.403853 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 398/800] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 20:00:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:24.404986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e02c18f-58bc-44c0-a464-24673b5b4abe None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:24.422683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37aed4-3c17-4948-a46f-11ee1f86df60 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:24.460638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28528853-e03f-44ee-b196-94e94c9bc981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:24.461410 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28528853-e03f-44ee-b196-94e94c9bc981 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:24.461784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28528853-e03f-44ee-b196-94e94c9bc981 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:24.462126 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28528853-e03f-44ee-b196-94e94c9bc981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:24.475091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28528853-e03f-44ee-b196-94e94c9bc981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:24.476494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28528853-e03f-44ee-b196-94e94c9bc981 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:24.493659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-28528853-e03f-44ee-b196-94e94c9bc981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:24.493659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:24.493659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:24.493659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:24.493659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:24.493659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:24.493659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:24.493659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:24.493659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:24.493659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:24.493659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:24.493659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:24.493659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:24.493659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:24.493659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:24.493659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:24.494817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:24.494817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:24.494817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:24.494817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:24.494817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:24.494817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:24.494817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:24.494817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:24.494817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:24.494817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:24.494817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:24.494817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:24.494817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:24.494817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:24.494817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:24.494817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:24.494817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:24.495953 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:24.495953 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:24.495953 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:24.495953 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 403/801] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:24.497351 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28528853-e03f-44ee-b196-94e94c9bc981 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:24.522081 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e02c18f-58bc-44c0-a464-24673b5b4abe None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:24.558510 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60d022-082b-4f5d-8440-437af7460361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:24.559144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60d022-082b-4f5d-8440-437af7460361 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:24.559425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60d022-082b-4f5d-8440-437af7460361 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:24.559694 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60d022-082b-4f5d-8440-437af7460361 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:24.567643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb60d022-082b-4f5d-8440-437af7460361 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:24.569080 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb60d022-082b-4f5d-8440-437af7460361 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:24.581134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-eb60d022-082b-4f5d-8440-437af7460361 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:24.581134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:24.581134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:24.581134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:24.581134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:24.581134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:24.581134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:24.581134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:24.581134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:24.581134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:24.581134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:24.581134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:24.581134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:24.581134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:24.581134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:24.582272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:24.582272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:24.582272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:24.582272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:24.582272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:24.582272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:24.582272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:24.582272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:24.582272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:24.582272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:24.582272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:24.582272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:24.582272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:24.582272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:24.582272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:24.582272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:24.582272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:24.588078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:24.588078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:24.588078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:24.588078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:24.588078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:24.588078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:24.588078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:24.588078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:24.588078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 399/802] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:24.588078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60d022-082b-4f5d-8440-437af7460361 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:24.607038 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28528853-e03f-44ee-b196-94e94c9bc981 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:24.634939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:24.649659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc4e11-93e7-4f8e-bc75-610985088722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:24.650315 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc4e11-93e7-4f8e-bc75-610985088722 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:24.650591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc4e11-93e7-4f8e-bc75-610985088722 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:24.650872 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc4e11-93e7-4f8e-bc75-610985088722 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:24.658789 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:24.664087 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc4e11-93e7-4f8e-bc75-610985088722 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:24.665527 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc4e11-93e7-4f8e-bc75-610985088722 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:24.681089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-83dc4e11-93e7-4f8e-bc75-610985088722 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:24.681089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:24.681089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:24.681089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:24.681089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:24.681089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:24.681089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:24.681089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:24.681089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:24.681089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:24.681089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:24.681089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:24.681089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:24.681089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:24.681089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:24.681089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:24.682176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:24.682176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:24.682176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:24.682176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:24.682176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:24.682176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:24.682176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:24.682176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:24.682176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:24.682176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:24.682176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:24.682176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:24.682176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:24.682176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:24.682176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:24.682176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:24.682176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:24.683086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:24.683086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:24.683086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:24.683086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 404/803] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:24.683086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc4e11-93e7-4f8e-bc75-610985088722 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:24.708010 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60d022-082b-4f5d-8440-437af7460361 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:24.749296 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d2cb9-2f62-4bfe-8169-74ba02a072b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:24.750291 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d2cb9-2f62-4bfe-8169-74ba02a072b7 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:24.750502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d2cb9-2f62-4bfe-8169-74ba02a072b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:24.750773 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d2cb9-2f62-4bfe-8169-74ba02a072b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:24.751414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d2cb9-2f62-4bfe-8169-74ba02a072b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:24.751713 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d2cb9-2f62-4bfe-8169-74ba02a072b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:24.753284 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d2cb9-2f62-4bfe-8169-74ba02a072b7 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:24.771566 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 400/804] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:24.771566 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d2cb9-2f62-4bfe-8169-74ba02a072b7 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:24.799537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc4e11-93e7-4f8e-bc75-610985088722 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:24.832623 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d7b1c-8163-46ad-9520-15d3ef4e0a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:24.833262 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d7b1c-8163-46ad-9520-15d3ef4e0a35 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:24.833544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d7b1c-8163-46ad-9520-15d3ef4e0a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:24.833846 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d7b1c-8163-46ad-9520-15d3ef4e0a35 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:24.834434 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41d7b1c-8163-46ad-9520-15d3ef4e0a35 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:24.835806 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41d7b1c-8163-46ad-9520-15d3ef4e0a35 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:24.845790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-f41d7b1c-8163-46ad-9520-15d3ef4e0a35 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:00:24.845790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:24.845790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:00:24.845790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:00:24.845790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:00:24.845790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:00:24.845790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:00:24.845790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:24.845790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:00:24.845790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:24.845790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:24.845790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:24.845790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:24.845790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:24.845790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:24.845790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:24.845790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:24.846817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:24.846817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:24.846817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:24.846817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:24.846817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:24.846817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:24.846817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:00:24.846817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:00:24.846817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:00:24.846817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:00:24.846817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:24.846817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:24.846817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:24.846817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:24.846817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:24.846817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:24.847682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:24.847682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:24.847682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:24.847682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:24.847682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:24.847682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:24.847682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:24.847682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:24.847682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:24.847682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:24.847682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:00:24.847682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:24.847682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:00:24.847682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:24.847682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 405/805] 10.222.0.73 () {56 vars in 1129 bytes} [Mon Aug 29 20:00:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:24.848468 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d7b1c-8163-46ad-9520-15d3ef4e0a35 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:24.887557 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:24.890485 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d2cb9-2f62-4bfe-8169-74ba02a072b7 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:24.912962 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:24.929859 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2579f4-6f0e-4ffc-a1c5-d60ebd0abcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:24.930477 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2579f4-6f0e-4ffc-a1c5-d60ebd0abcc8 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:24.930754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2579f4-6f0e-4ffc-a1c5-d60ebd0abcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:24.931025 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2579f4-6f0e-4ffc-a1c5-d60ebd0abcc8 None admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:24.931711 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2579f4-6f0e-4ffc-a1c5-d60ebd0abcc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:24.932012 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2579f4-6f0e-4ffc-a1c5-d60ebd0abcc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:24.933430 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2579f4-6f0e-4ffc-a1c5-d60ebd0abcc8 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:24.946318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 401/806] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:00:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:24.947248 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2579f4-6f0e-4ffc-a1c5-d60ebd0abcc8 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:24.968462 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d7b1c-8163-46ad-9520-15d3ef4e0a35 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:25.008460 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e19bf-fcd5-43a4-86ad-df3098ae2720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:25.008979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e19bf-fcd5-43a4-86ad-df3098ae2720 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:25.009345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e19bf-fcd5-43a4-86ad-df3098ae2720 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:25.009541 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e19bf-fcd5-43a4-86ad-df3098ae2720 None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:25.010189 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e19bf-fcd5-43a4-86ad-df3098ae2720 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:25.010681 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e19bf-fcd5-43a4-86ad-df3098ae2720 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:25.011840 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e19bf-fcd5-43a4-86ad-df3098ae2720 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:25.025424 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 406/807] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:25.026294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e19bf-fcd5-43a4-86ad-df3098ae2720 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:25.056525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2579f4-6f0e-4ffc-a1c5-d60ebd0abcc8 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:25.091019 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e23207-2cb5-4d0c-a138-0438c392196c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:25.092161 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e23207-2cb5-4d0c-a138-0438c392196c None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:25.092435 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e23207-2cb5-4d0c-a138-0438c392196c None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:25.092744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e23207-2cb5-4d0c-a138-0438c392196c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:25.103962 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e23207-2cb5-4d0c-a138-0438c392196c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:25.105399 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e23207-2cb5-4d0c-a138-0438c392196c None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:25.118344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-f7e23207-2cb5-4d0c-a138-0438c392196c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:00:25.118344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:25.118344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:25.118344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:25.118344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:25.118344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:25.118344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:25.118344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:25.118344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:25.118344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:25.118344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:25.118344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:25.118344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:00:25.118344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:00:25.118344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:00:25.118344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:00:25.119496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:25.119496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:25.119496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:25.119496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:25.119496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:25.119496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:25.119496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:25.119496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:25.119496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:25.119496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:25.119496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:25.119496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:25.119496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:25.119496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:25.119496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:25.119496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:25.119496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:25.119496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:25.120618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:25.120618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:25.120618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:00:25.120618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:00:25.120618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:00:25.120618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:00:25.120618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:00:25.120618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:00:25.120618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:00:25.120618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:25.120618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 402/808] 10.222.0.73 () {56 vars in 1117 bytes} [Mon Aug 29 20:00:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:25.120618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e23207-2cb5-4d0c-a138-0438c392196c None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:25.136788 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e19bf-fcd5-43a4-86ad-df3098ae2720 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:25.143676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:25.152395 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:25.173478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382c501-ea03-4483-9733-5869051fcaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:25.174113 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382c501-ea03-4483-9733-5869051fcaf9 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:25.174391 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382c501-ea03-4483-9733-5869051fcaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:25.174658 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382c501-ea03-4483-9733-5869051fcaf9 None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:25.175307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382c501-ea03-4483-9733-5869051fcaf9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:25.175606 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382c501-ea03-4483-9733-5869051fcaf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:25.177051 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382c501-ea03-4483-9733-5869051fcaf9 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:25.181416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 407/809] 10.222.0.73 () {56 vars in 1124 bytes} [Mon Aug 29 20:00:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:25.182583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269aec57-6bb7-46f7-8fe4-5a1168397185 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:25.182860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269aec57-6bb7-46f7-8fe4-5a1168397185 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:25.183128 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269aec57-6bb7-46f7-8fe4-5a1168397185 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:25.184482 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 408/810] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:25.184970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269aec57-6bb7-46f7-8fe4-5a1168397185 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:25.231669 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e23207-2cb5-4d0c-a138-0438c392196c None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:25.267781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186dcd9b-0949-4521-94ad-106b251c2810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:25.268711 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186dcd9b-0949-4521-94ad-106b251c2810 None admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:25.268993 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186dcd9b-0949-4521-94ad-106b251c2810 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:25.269267 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186dcd9b-0949-4521-94ad-106b251c2810 None admin] PATH_INFO: `/v3/projects/2f5ec615d98548159487c97cead336a0/users/b92e30da8f394f3abc19b9e86ef126e5/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:25.271309 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186dcd9b-0949-4521-94ad-106b251c2810 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f5ec615d98548159487c97cead336a0, user_id=b92e30da8f394f3abc19b9e86ef126e5, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:25.272673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186dcd9b-0949-4521-94ad-106b251c2810 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:25.285979 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 403/811] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 20:00:25 2022] PUT /identity/v3/projects/2f5ec615d98548159487c97cead336a0/users/b92e30da8f394f3abc19b9e86ef126e5/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:00:25.287004 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186dcd9b-0949-4521-94ad-106b251c2810 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:25.288056 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269aec57-6bb7-46f7-8fe4-5a1168397185 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:25.330921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647bae93-c16d-42a4-9929-d35b114ef478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:25.331585 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bae93-c16d-42a4-9929-d35b114ef478 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:25.331871 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bae93-c16d-42a4-9929-d35b114ef478 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:25.332147 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bae93-c16d-42a4-9929-d35b114ef478 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:25.341685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647bae93-c16d-42a4-9929-d35b114ef478 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:25.344007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647bae93-c16d-42a4-9929-d35b114ef478 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:25.357137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-647bae93-c16d-42a4-9929-d35b114ef478 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:00:25.357137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:25.357137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:25.357137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:25.357137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:25.357137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:25.357137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:25.357137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:25.357137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:25.357137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:25.357137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:25.357137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:25.357137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:00:25.357137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:00:25.357137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:00:25.357137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:00:25.358777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:25.358777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:25.358777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:25.358777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:25.358777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:25.358777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:25.358777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:25.358777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:25.358777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:25.358777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:25.358777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:25.358777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:25.358777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:25.358777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:25.358777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:25.358777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:25.358777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:25.358777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:25.360376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:25.360376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:25.360376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:00:25.360376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:00:25.360376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:00:25.360376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:00:25.360376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:00:25.360376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:00:25.360376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:00:25.360376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:25.360376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 409/812] 10.222.0.73 () {56 vars in 1093 bytes} [Mon Aug 29 20:00:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:25.360376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647bae93-c16d-42a4-9929-d35b114ef478 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:25.391079 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:25.407652 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:25.419454 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186dcd9b-0949-4521-94ad-106b251c2810 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:25.460359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5fe83a-4124-4f49-a341-5fbf362639c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:25.461127 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fe83a-4124-4f49-a341-5fbf362639c9 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:25.461486 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fe83a-4124-4f49-a341-5fbf362639c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:25.461843 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fe83a-4124-4f49-a341-5fbf362639c9 None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:25.462649 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5fe83a-4124-4f49-a341-5fbf362639c9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:25.462965 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5fe83a-4124-4f49-a341-5fbf362639c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:25.464620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5fe83a-4124-4f49-a341-5fbf362639c9 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:25.483273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647bae93-c16d-42a4-9929-d35b114ef478 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:25.484538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 404/813] 10.222.0.73 () {56 vars in 1149 bytes} [Mon Aug 29 20:00:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:25.489376 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed70fc9-3df0-45e3-b99f-9712172e39bc None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:25.489376 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed70fc9-3df0-45e3-b99f-9712172e39bc None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:25.489376 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed70fc9-3df0-45e3-b99f-9712172e39bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:25.505142 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-4ed70fc9-3df0-45e3-b99f-9712172e39bc None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:25.506510 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-4ed70fc9-3df0-45e3-b99f-9712172e39bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:25.521588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245dc935-266f-45e1-82a3-710a71118e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:25.522156 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245dc935-266f-45e1-82a3-710a71118e56 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:25.522434 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245dc935-266f-45e1-82a3-710a71118e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:25.522711 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245dc935-266f-45e1-82a3-710a71118e56 None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:25.523393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245dc935-266f-45e1-82a3-710a71118e56 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:25.523791 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245dc935-266f-45e1-82a3-710a71118e56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:25.525161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245dc935-266f-45e1-82a3-710a71118e56 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:25.533886 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 410/814] 10.222.0.73 () {56 vars in 1124 bytes} [Mon Aug 29 20:00:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:25.535136 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245dc935-266f-45e1-82a3-710a71118e56 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:25.582241 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 405/815] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:25.582241 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed70fc9-3df0-45e3-b99f-9712172e39bc None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:25.655073 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245dc935-266f-45e1-82a3-710a71118e56 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:25.656628 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:25.679663 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:25.696917 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97467690-8ba4-43ff-99e1-c0447935e0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:25.697899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97467690-8ba4-43ff-99e1-c0447935e0af None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:25.698344 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97467690-8ba4-43ff-99e1-c0447935e0af None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:25.698893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97467690-8ba4-43ff-99e1-c0447935e0af None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:25.699903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97467690-8ba4-43ff-99e1-c0447935e0af None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:25.700215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97467690-8ba4-43ff-99e1-c0447935e0af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:25.702425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97467690-8ba4-43ff-99e1-c0447935e0af None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:25.722763 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 411/816] 10.222.0.73 () {56 vars in 1133 bytes} [Mon Aug 29 20:00:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:25.723431 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed70fc9-3df0-45e3-b99f-9712172e39bc None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:25.723684 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97467690-8ba4-43ff-99e1-c0447935e0af None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:25.766530 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be4b31-e52c-4f69-80ab-f6479d16ed16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:25.767248 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be4b31-e52c-4f69-80ab-f6479d16ed16 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:25.767493 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be4b31-e52c-4f69-80ab-f6479d16ed16 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:25.767776 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be4b31-e52c-4f69-80ab-f6479d16ed16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:25.778853 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4be4b31-e52c-4f69-80ab-f6479d16ed16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:25.780581 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4be4b31-e52c-4f69-80ab-f6479d16ed16 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:25.796419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-a4be4b31-e52c-4f69-80ab-f6479d16ed16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:25.796419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:25.796419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:25.796419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:25.796419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:25.796419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:25.796419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:25.796419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:25.796419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:25.796419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:25.796419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:25.796419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:25.796419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:25.796419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:25.796419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:25.796419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:25.797420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:25.797420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:25.797420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:25.797420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:25.797420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:25.797420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:25.797420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:25.797420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:25.797420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:25.797420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:25.797420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:25.797420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:25.797420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:25.797420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:25.797420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:25.797420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:25.797420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:25.798956 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:25.798956 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:25.798956 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:25.798956 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 406/817] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:25.798956 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be4b31-e52c-4f69-80ab-f6479d16ed16 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:25.841014 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97467690-8ba4-43ff-99e1-c0447935e0af None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:25.878652 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9e6a8-21ee-44c2-870e-0fa0a61812b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:25.879442 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9e6a8-21ee-44c2-870e-0fa0a61812b6 None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:25.879820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9e6a8-21ee-44c2-870e-0fa0a61812b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:25.880195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9e6a8-21ee-44c2-870e-0fa0a61812b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:25.880961 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee9e6a8-21ee-44c2-870e-0fa0a61812b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:25.882512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee9e6a8-21ee-44c2-870e-0fa0a61812b6 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:25.909759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 412/818] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:00:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:00:25.909759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9e6a8-21ee-44c2-870e-0fa0a61812b6 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:25.912773 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be4b31-e52c-4f69-80ab-f6479d16ed16 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:25.933803 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:25.940963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:25.963437 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a1ebf-f3ab-43f3-adcb-9396e2ffecb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:25.964082 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a1ebf-f3ab-43f3-adcb-9396e2ffecb2 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:25.964366 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a1ebf-f3ab-43f3-adcb-9396e2ffecb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:25.964644 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a1ebf-f3ab-43f3-adcb-9396e2ffecb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:26.033092 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9e6a8-21ee-44c2-870e-0fa0a61812b6 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:26.047859 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a1ebf-f3ab-43f3-adcb-9396e2ffecb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:26.049453 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a1ebf-f3ab-43f3-adcb-9396e2ffecb2 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:26.077078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79450f-bebd-45b4-a919-4eba4662dcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:26.078045 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79450f-bebd-45b4-a919-4eba4662dcf4 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:26.078521 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79450f-bebd-45b4-a919-4eba4662dcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:26.079063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79450f-bebd-45b4-a919-4eba4662dcf4 None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:26.079850 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c79450f-bebd-45b4-a919-4eba4662dcf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:26.080310 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c79450f-bebd-45b4-a919-4eba4662dcf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:26.085367 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c79450f-bebd-45b4-a919-4eba4662dcf4 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:26.099530 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 413/819] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:26.100878 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79450f-bebd-45b4-a919-4eba4662dcf4 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:26.187620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 407/820] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:00:25 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:00:26.188781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a1ebf-f3ab-43f3-adcb-9396e2ffecb2 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:26.207581 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:26.207868 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:26.240659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79450f-bebd-45b4-a919-4eba4662dcf4 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:26.278955 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a0c3d-a550-4069-bb60-7f3b509c08f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:26.280602 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a0c3d-a550-4069-bb60-7f3b509c08f5 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:26.281161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a0c3d-a550-4069-bb60-7f3b509c08f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:26.281650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a0c3d-a550-4069-bb60-7f3b509c08f5 None admin] PATH_INFO: `/v3/services/network` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:26.282554 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a0c3d-a550-4069-bb60-7f3b509c08f5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:26.284211 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a0c3d-a550-4069-bb60-7f3b509c08f5 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:26.294004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-037a0c3d-a550-4069-bb60-7f3b509c08f5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:00:26.294004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:26.294004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:00:26.294004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:00:26.294004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:00:26.294004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:00:26.294004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:00:26.294004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:26.294004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:00:26.294004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:26.294004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:26.294004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:26.294004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:26.294004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:26.294004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:26.294004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:26.294004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:26.295661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:26.295661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:26.295661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:26.295661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:26.295661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:26.295661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:26.295661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:00:26.295661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:00:26.295661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:00:26.295661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:00:26.295661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:26.295661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:26.295661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:26.295661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:26.295661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:26.295661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:26.296493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:26.296493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:26.296493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:26.296493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:26.296493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:26.296493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:26.296493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:26.296493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:26.296493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:26.296493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:26.296493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:00:26.296493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:26.296493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:00:26.296493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:26.303922 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 414/821] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:26 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:26.305102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a0c3d-a550-4069-bb60-7f3b509c08f5 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:26.309819 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a1ebf-f3ab-43f3-adcb-9396e2ffecb2 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:26.347627 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cae6d3-af66-4818-9ac8-f7c4064033bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:26.348019 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cae6d3-af66-4818-9ac8-f7c4064033bb None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:26.348285 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cae6d3-af66-4818-9ac8-f7c4064033bb None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:26.348556 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cae6d3-af66-4818-9ac8-f7c4064033bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:26.356194 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cae6d3-af66-4818-9ac8-f7c4064033bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:26.358335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cae6d3-af66-4818-9ac8-f7c4064033bb None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:26.370470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-36cae6d3-af66-4818-9ac8-f7c4064033bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:26.370470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:26.370470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:26.370470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:26.370470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:26.370470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:26.370470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:26.370470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:26.370470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:26.370470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:26.370470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:26.370470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:26.370470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:26.370470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:26.370470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:26.371389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:26.371389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:26.371389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:26.371389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:26.371389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:26.371389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:26.371389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:26.371389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:26.371389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:26.371389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:26.371389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:26.371389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:26.371389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:26.371389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:26.371389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:26.371389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:26.371389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:26.372278 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:26.372278 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:26.372278 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:26.372278 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:26.372278 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:26.372278 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:26.372278 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:26.372278 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:26.376922 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 408/822] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:26.377886 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cae6d3-af66-4818-9ac8-f7c4064033bb None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:26.427999 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a0c3d-a550-4069-bb60-7f3b509c08f5 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:26.463903 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:26.469163 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba18cf2-b376-465e-b71e-d0408be50ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:26.469832 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba18cf2-b376-465e-b71e-d0408be50ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:26.470110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba18cf2-b376-465e-b71e-d0408be50ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:26.470378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba18cf2-b376-465e-b71e-d0408be50ed7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:26.479728 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:26.484241 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba18cf2-b376-465e-b71e-d0408be50ed7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:26.485754 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba18cf2-b376-465e-b71e-d0408be50ed7 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:26.492489 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cae6d3-af66-4818-9ac8-f7c4064033bb None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:26.501472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-7ba18cf2-b376-465e-b71e-d0408be50ed7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:26.501472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:26.501472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:26.501472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:26.501472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:26.501472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:26.501472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:26.501472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:26.501472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:26.501472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:26.501472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:26.501472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:26.501472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:26.501472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:26.501472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:26.501472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:26.502697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:26.502697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:26.502697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:26.502697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:26.502697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:26.502697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:26.502697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:26.502697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:26.502697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:26.502697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:26.502697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:26.502697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:26.502697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:26.502697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:26.502697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:26.502697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:26.502697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:26.503763 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:26.503763 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:26.503763 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:26.507326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 415/823] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:26.508176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba18cf2-b376-465e-b71e-d0408be50ed7 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:26.535005 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cb5aa-3a6d-4617-a971-0f2dd45b8721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:26.535852 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cb5aa-3a6d-4617-a971-0f2dd45b8721 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:26.536306 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cb5aa-3a6d-4617-a971-0f2dd45b8721 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:26.536727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cb5aa-3a6d-4617-a971-0f2dd45b8721 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:26.537513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cb5aa-3a6d-4617-a971-0f2dd45b8721 None admin] RBAC: Adding query filter params (name=network) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:26.537996 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cb5aa-3a6d-4617-a971-0f2dd45b8721 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:26.539642 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cb5aa-3a6d-4617-a971-0f2dd45b8721 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:26.545675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 409/824] 10.222.0.73 () {56 vars in 1112 bytes} [Mon Aug 29 20:00:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:26.546710 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cb5aa-3a6d-4617-a971-0f2dd45b8721 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:26.614970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba18cf2-b376-465e-b71e-d0408be50ed7 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:26.650904 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51ec1f-2f22-4862-8c05-7cf0e3f4eec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:26.651511 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51ec1f-2f22-4862-8c05-7cf0e3f4eec3 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:26.651794 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51ec1f-2f22-4862-8c05-7cf0e3f4eec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:26.652055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51ec1f-2f22-4862-8c05-7cf0e3f4eec3 None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:26.652746 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e51ec1f-2f22-4862-8c05-7cf0e3f4eec3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:26.653045 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e51ec1f-2f22-4862-8c05-7cf0e3f4eec3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:26.654574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e51ec1f-2f22-4862-8c05-7cf0e3f4eec3 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:26.657689 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cb5aa-3a6d-4617-a971-0f2dd45b8721 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:26.673059 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 416/825] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:00:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:26.673751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51ec1f-2f22-4862-8c05-7cf0e3f4eec3 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:26.695438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce92b7-8b26-4cd3-afe2-b86d1d9a5f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:26.696403 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce92b7-8b26-4cd3-afe2-b86d1d9a5f82 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:26.696888 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce92b7-8b26-4cd3-afe2-b86d1d9a5f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:26.697408 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce92b7-8b26-4cd3-afe2-b86d1d9a5f82 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:26.698284 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ce92b7-8b26-4cd3-afe2-b86d1d9a5f82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:26.698787 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ce92b7-8b26-4cd3-afe2-b86d1d9a5f82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:26.700544 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ce92b7-8b26-4cd3-afe2-b86d1d9a5f82 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:26.711595 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:26.715995 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 410/826] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:26.717067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce92b7-8b26-4cd3-afe2-b86d1d9a5f82 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:26.737512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:26.797579 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51ec1f-2f22-4862-8c05-7cf0e3f4eec3 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:26.825257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce92b7-8b26-4cd3-afe2-b86d1d9a5f82 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:26.835547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30b603-9a71-4460-8f35-9bed9962cb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:26.836160 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30b603-9a71-4460-8f35-9bed9962cb15 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:26.836435 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30b603-9a71-4460-8f35-9bed9962cb15 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:26.836758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30b603-9a71-4460-8f35-9bed9962cb15 None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:26.837528 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30b603-9a71-4460-8f35-9bed9962cb15 None admin] RBAC: Adding query filter params (type=network) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:26.837845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30b603-9a71-4460-8f35-9bed9962cb15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:26.839492 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30b603-9a71-4460-8f35-9bed9962cb15 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:26.845926 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 417/827] 10.222.0.73 () {56 vars in 1112 bytes} [Mon Aug 29 20:00:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:26.846897 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30b603-9a71-4460-8f35-9bed9962cb15 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:26.867979 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbddaa9e-2b46-4aa6-ab63-c4a521dc14f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:26.868651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddaa9e-2b46-4aa6-ab63-c4a521dc14f7 None admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:26.868932 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddaa9e-2b46-4aa6-ab63-c4a521dc14f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:26.869208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddaa9e-2b46-4aa6-ab63-c4a521dc14f7 None admin] PATH_INFO: `/v3/projects/2f5ec615d98548159487c97cead336a0/users/e0c76478b6694782b896cdf9cda32d9c/roles/8ca2ec70f49a4f4da3ce9f3ac0b73083` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:26.888341 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbddaa9e-2b46-4aa6-ab63-c4a521dc14f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f5ec615d98548159487c97cead336a0, user_id=e0c76478b6694782b896cdf9cda32d9c, role_id=8ca2ec70f49a4f4da3ce9f3ac0b73083)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:26.891868 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbddaa9e-2b46-4aa6-ab63-c4a521dc14f7 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:26.909336 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 411/828] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 20:00:26 2022] PUT /identity/v3/projects/2f5ec615d98548159487c97cead336a0/users/e0c76478b6694782b896cdf9cda32d9c/roles/8ca2ec70f49a4f4da3ce9f3ac0b73083 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:00:26.910026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbddaa9e-2b46-4aa6-ab63-c4a521dc14f7 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:26.970885 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:27.000054 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30b603-9a71-4460-8f35-9bed9962cb15 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:27.029952 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:27.046955 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbddaa9e-2b46-4aa6-ab63-c4a521dc14f7 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:27.094645 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db17e09-b44d-46ea-a06c-c3b130fca1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:27.095278 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db17e09-b44d-46ea-a06c-c3b130fca1cd None admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:27.095558 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db17e09-b44d-46ea-a06c-c3b130fca1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:27.095834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db17e09-b44d-46ea-a06c-c3b130fca1cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:27.096643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db17e09-b44d-46ea-a06c-c3b130fca1cd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:27.098061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db17e09-b44d-46ea-a06c-c3b130fca1cd None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:27.100896 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a10b3-b9e9-42b4-9d0a-151c5523c004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:27.101646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a10b3-b9e9-42b4-9d0a-151c5523c004 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:27.102004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a10b3-b9e9-42b4-9d0a-151c5523c004 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:27.102284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a10b3-b9e9-42b4-9d0a-151c5523c004 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:27.111012 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a10b3-b9e9-42b4-9d0a-151c5523c004 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:27.112557 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a10b3-b9e9-42b4-9d0a-151c5523c004 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:27.128214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-023a10b3-b9e9-42b4-9d0a-151c5523c004 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:27.128214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:27.128214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:27.128214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:27.128214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:27.128214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:27.128214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:27.128214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:27.128214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:27.128214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:27.128214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:27.128214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:27.128214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:27.128214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:27.128214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:27.129255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:27.129255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:27.129255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:27.129255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:27.129255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:27.129255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:27.129255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:27.129255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:27.129255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:27.129255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:27.129255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:27.129255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:27.129255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:27.129255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:27.129255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:27.129255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:27.129255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:27.130182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:27.130182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:27.130182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:27.130182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:27.130182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:27.130182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:27.130182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:27.130182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:27.130182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 418/829] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:27.136920 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a10b3-b9e9-42b4-9d0a-151c5523c004 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:27.153871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 412/830] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:00:26 2022] POST /identity/v3/endpoints => generated 337 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:00:27.277518 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a10b3-b9e9-42b4-9d0a-151c5523c004 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:27.340745 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd73c6-2243-4fb4-84b7-9a61046c9a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:27.341817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd73c6-2243-4fb4-84b7-9a61046c9a18 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:27.342303 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd73c6-2243-4fb4-84b7-9a61046c9a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:27.342865 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd73c6-2243-4fb4-84b7-9a61046c9a18 None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:27.343892 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd73c6-2243-4fb4-84b7-9a61046c9a18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:27.344414 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd73c6-2243-4fb4-84b7-9a61046c9a18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:27.346157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd73c6-2243-4fb4-84b7-9a61046c9a18 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:27.360262 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 419/831] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:00:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:27.366041 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db17e09-b44d-46ea-a06c-c3b130fca1cd None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:27.366768 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05da85-6848-4f5e-9be3-46f4a64ebec1 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:27.367055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05da85-6848-4f5e-9be3-46f4a64ebec1 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:27.367337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05da85-6848-4f5e-9be3-46f4a64ebec1 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:27.369072 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 420/832] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:00:27.375470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b611452-9862-4d55-9bc4-48e11a0e4b1f None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:27.375786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b611452-9862-4d55-9bc4-48e11a0e4b1f None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:27.376067 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b611452-9862-4d55-9bc4-48e11a0e4b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:27.398457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-8b611452-9862-4d55-9bc4-48e11a0e4b1f None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:27.399833 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-8b611452-9862-4d55-9bc4-48e11a0e4b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:27.441633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:27.502842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 421/833] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:27.521369 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db17e09-b44d-46ea-a06c-c3b130fca1cd None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:27.528648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:27.559225 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf38e05-4037-4cbb-b76c-f157c06c13d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:27.559933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf38e05-4037-4cbb-b76c-f157c06c13d1 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:27.560283 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf38e05-4037-4cbb-b76c-f157c06c13d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:27.561344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf38e05-4037-4cbb-b76c-f157c06c13d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:27.562228 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf38e05-4037-4cbb-b76c-f157c06c13d1 None admin] RBAC: Adding query filter params (role.id=296ee2c194824485af71618619f82e09, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=95cf77a92a174ff8b0503baf5c7e63be) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:27.562607 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf38e05-4037-4cbb-b76c-f157c06c13d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:27.564238 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf38e05-4037-4cbb-b76c-f157c06c13d1 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:27.571407 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 413/834] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:00:27 2022] GET /identity/v3/role_assignments?user.id=95cf77a92a174ff8b0503baf5c7e63be&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=296ee2c194824485af71618619f82e09 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:27.785207 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b611452-9862-4d55-9bc4-48e11a0e4b1f None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:27.894182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b611452-9862-4d55-9bc4-48e11a0e4b1f None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:27.932151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92881d15-6bbe-4622-9e6c-eab7ab1bc996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:27.932789 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92881d15-6bbe-4622-9e6c-eab7ab1bc996 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:27.933069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92881d15-6bbe-4622-9e6c-eab7ab1bc996 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:27.933341 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92881d15-6bbe-4622-9e6c-eab7ab1bc996 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:27.944384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92881d15-6bbe-4622-9e6c-eab7ab1bc996 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:00:27.949325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-92881d15-6bbe-4622-9e6c-eab7ab1bc996 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:00:27.949325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:27.949325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:27.949325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:27.949325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:27.949325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:27.949325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:27.949325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:27.949325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:27.949325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:27.949325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:27.949325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:27.949325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:00:27.949325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:00:27.949325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:00:27.949325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:00:27.950293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:00:27.950293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:00:27.950293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:00:27.950293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:00:27.950293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:27.950293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:27.950293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:27.950293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:27.950293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:27.950293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:27.950293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:27.950293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:27.950293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:27.950293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:27.950293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:27.950293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:27.951475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:27.951475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:27.951475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:27.951475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:27.951475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:00:27.951475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:00:27.951475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:00:27.951475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:00:27.951475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:00:27.951475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:00:27.951475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:00:27.951475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:27.951475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 422/835] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 20:00:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:27.958822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf38e05-4037-4cbb-b76c-f157c06c13d1 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:28.071861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf38e05-4037-4cbb-b76c-f157c06c13d1 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:28.109975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7080dc-29bc-4cf9-987c-aa13fadebbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:28.110594 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7080dc-29bc-4cf9-987c-aa13fadebbfd None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:28.110876 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7080dc-29bc-4cf9-987c-aa13fadebbfd None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:28.111151 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7080dc-29bc-4cf9-987c-aa13fadebbfd None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:28.111825 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7080dc-29bc-4cf9-987c-aa13fadebbfd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:28.112132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7080dc-29bc-4cf9-987c-aa13fadebbfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:28.113495 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7080dc-29bc-4cf9-987c-aa13fadebbfd None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:28.122636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 414/836] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 20:00:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:28.125145 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe036f7-53fe-4cf4-9de4-7b4d2ca710b3 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:28.125441 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe036f7-53fe-4cf4-9de4-7b4d2ca710b3 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:28.125736 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe036f7-53fe-4cf4-9de4-7b4d2ca710b3 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:28.127259 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 423/837] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:28.132108 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7080dc-29bc-4cf9-987c-aa13fadebbfd None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:28.134940 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5716ec4-28c9-41c1-a9f7-713b2b7ff5c6 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:28.135231 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5716ec4-28c9-41c1-a9f7-713b2b7ff5c6 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:28.135512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5716ec4-28c9-41c1-a9f7-713b2b7ff5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:28.155718 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-d5716ec4-28c9-41c1-a9f7-713b2b7ff5c6 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:28.157032 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-d5716ec4-28c9-41c1-a9f7-713b2b7ff5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:28.166735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:28.194290 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:28.252242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7080dc-29bc-4cf9-987c-aa13fadebbfd None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:28.265174 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 424/838] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:28.290144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63442bf-dcab-4b40-89ce-449bf25f7b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:28.291221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63442bf-dcab-4b40-89ce-449bf25f7b51 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:28.291716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63442bf-dcab-4b40-89ce-449bf25f7b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:28.292192 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63442bf-dcab-4b40-89ce-449bf25f7b51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:28.304175 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63442bf-dcab-4b40-89ce-449bf25f7b51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:28.305900 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63442bf-dcab-4b40-89ce-449bf25f7b51 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:28.321847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-c63442bf-dcab-4b40-89ce-449bf25f7b51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:28.321847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:28.321847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:28.321847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:28.321847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:28.321847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:28.321847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:28.321847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:28.321847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:28.321847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:28.321847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:28.321847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:28.321847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:28.321847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:28.321847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:28.321847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:28.322861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:28.322861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:28.322861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:28.322861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:28.322861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:28.322861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:28.322861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:28.322861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:28.322861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:28.322861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:28.322861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:28.322861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:28.322861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:28.322861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:28.322861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:28.322861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:28.322861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:28.323773 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:28.323773 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:28.323773 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:28.325846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 415/839] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:28.334923 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5716ec4-28c9-41c1-a9f7-713b2b7ff5c6 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:28.446576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5716ec4-28c9-41c1-a9f7-713b2b7ff5c6 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:28.483853 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b17b9-7710-48a6-aa1d-d26a1e37794b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:28.484840 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b17b9-7710-48a6-aa1d-d26a1e37794b None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:28.485339 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b17b9-7710-48a6-aa1d-d26a1e37794b None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:28.485846 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b17b9-7710-48a6-aa1d-d26a1e37794b None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:28.486719 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b17b9-7710-48a6-aa1d-d26a1e37794b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:28.487226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b17b9-7710-48a6-aa1d-d26a1e37794b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:28.488796 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b17b9-7710-48a6-aa1d-d26a1e37794b None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:28.502716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 425/840] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:28.508166 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63442bf-dcab-4b40-89ce-449bf25f7b51 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:28.533406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b17b9-7710-48a6-aa1d-d26a1e37794b None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:28.550276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:28.623041 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:28.646090 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63442bf-dcab-4b40-89ce-449bf25f7b51 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:28.664744 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b17b9-7710-48a6-aa1d-d26a1e37794b None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:28.689727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0233acc-0588-4bb4-8adb-2a43c15b8dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:28.690380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0233acc-0588-4bb4-8adb-2a43c15b8dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:28.690820 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0233acc-0588-4bb4-8adb-2a43c15b8dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:28.691218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0233acc-0588-4bb4-8adb-2a43c15b8dc6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:28.703245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0233acc-0588-4bb4-8adb-2a43c15b8dc6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:28.704574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7351029-38ee-47c3-b381-bf78f898ed06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:28.704804 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0233acc-0588-4bb4-8adb-2a43c15b8dc6 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:28.705396 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7351029-38ee-47c3-b381-bf78f898ed06 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:28.705789 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7351029-38ee-47c3-b381-bf78f898ed06 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:28.706036 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7351029-38ee-47c3-b381-bf78f898ed06 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:28.706669 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7351029-38ee-47c3-b381-bf78f898ed06 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:28.708016 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7351029-38ee-47c3-b381-bf78f898ed06 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:28.718215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-e7351029-38ee-47c3-b381-bf78f898ed06 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:00:28.718215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:28.718215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:00:28.718215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:00:28.718215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:00:28.718215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:00:28.718215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:00:28.718215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:28.718215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:00:28.718215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:28.718215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:28.718215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:28.718215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:28.718215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:28.718215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:28.718215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:28.718215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:28.719318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:28.719318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:28.719318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:28.719318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:28.719318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:28.719318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:28.719318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:00:28.719318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:00:28.719318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:00:28.719318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:00:28.719318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:28.719318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:28.719318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:28.719318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:28.719318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:28.719318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:28.721161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:28.721161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:28.721161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:28.721161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:28.721161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:28.721161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:28.721161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:28.721161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:28.721161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:28.721161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:28.721161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:00:28.721161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:28.721161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:00:28.721161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:28.723042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-e0233acc-0588-4bb4-8adb-2a43c15b8dc6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:00:28.723042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:28.723042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:28.723042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:28.723042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:28.723042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:28.723042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:28.723042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:28.723042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:28.723042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:28.723042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:28.723042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:28.723042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:00:28.723042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:00:28.723042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:00:28.723042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:00:28.726327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:28.726327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:28.726327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:28.726327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:28.726327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:28.726327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:28.726327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:28.726327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:28.726327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:28.726327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:28.726327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:28.726327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:28.726327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:28.726327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:28.726327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:28.726327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:28.726327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:28.726327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:28.727258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:28.727258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:28.727258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:00:28.727258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:00:28.727258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:00:28.727258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:00:28.727258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:00:28.727258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:00:28.727258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:00:28.727258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:28.727258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 416/842] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 20:00:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:28.727842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 426/842] 10.222.0.73 () {56 vars in 1114 bytes} [Mon Aug 29 20:00:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:00:28.727910 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0233acc-0588-4bb4-8adb-2a43c15b8dc6 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:28.728676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7351029-38ee-47c3-b381-bf78f898ed06 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:28.833475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:28.857340 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0233acc-0588-4bb4-8adb-2a43c15b8dc6 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:28.864194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7351029-38ee-47c3-b381-bf78f898ed06 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:28.902824 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c81ac-cd8b-4253-9212-ce01496e533d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:28.903863 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c81ac-cd8b-4253-9212-ce01496e533d None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:28.904392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2770d9a-150a-47e1-9db4-6bad3915fd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:28.904719 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c81ac-cd8b-4253-9212-ce01496e533d None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:28.905008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2770d9a-150a-47e1-9db4-6bad3915fd43 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:28.905291 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2770d9a-150a-47e1-9db4-6bad3915fd43 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:28.905570 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2770d9a-150a-47e1-9db4-6bad3915fd43 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:28.905849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c81ac-cd8b-4253-9212-ce01496e533d None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:28.906236 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2770d9a-150a-47e1-9db4-6bad3915fd43 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:28.906548 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2770d9a-150a-47e1-9db4-6bad3915fd43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:28.907221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c81ac-cd8b-4253-9212-ce01496e533d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:28.907726 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c81ac-cd8b-4253-9212-ce01496e533d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:28.907903 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2770d9a-150a-47e1-9db4-6bad3915fd43 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:28.909481 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c81ac-cd8b-4253-9212-ce01496e533d None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:28.918562 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 417/843] 10.222.0.73 () {56 vars in 1114 bytes} [Mon Aug 29 20:00:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:28.919631 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2770d9a-150a-47e1-9db4-6bad3915fd43 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:28.933334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:28.941809 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 427/844] 10.222.0.73 () {56 vars in 1135 bytes} [Mon Aug 29 20:00:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:28.946111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c81ac-cd8b-4253-9212-ce01496e533d None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:29.049324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2770d9a-150a-47e1-9db4-6bad3915fd43 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:29.087331 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0400e8c-53c0-47fa-945d-d43baf3cd96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:29.088316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0400e8c-53c0-47fa-945d-d43baf3cd96b None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:29.088816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0400e8c-53c0-47fa-945d-d43baf3cd96b None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:29.088923 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c81ac-cd8b-4253-9212-ce01496e533d None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:29.089382 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0400e8c-53c0-47fa-945d-d43baf3cd96b None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:29.090233 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0400e8c-53c0-47fa-945d-d43baf3cd96b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:29.091405 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0400e8c-53c0-47fa-945d-d43baf3cd96b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:29.093810 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0400e8c-53c0-47fa-945d-d43baf3cd96b None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:29.106066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 418/845] 10.222.0.73 () {56 vars in 1114 bytes} [Mon Aug 29 20:00:28 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:29.130756 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941713e-c724-4d9c-b552-310d0050b8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:29.131389 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941713e-c724-4d9c-b552-310d0050b8ed None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:29.131671 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941713e-c724-4d9c-b552-310d0050b8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:29.131944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941713e-c724-4d9c-b552-310d0050b8ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:29.134961 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:29.216031 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b71f66-dbff-4ac5-a82d-86326b641b56 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:29.216414 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941713e-c724-4d9c-b552-310d0050b8ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:29.217640 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b71f66-dbff-4ac5-a82d-86326b641b56 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:29.218144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b71f66-dbff-4ac5-a82d-86326b641b56 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:29.218656 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941713e-c724-4d9c-b552-310d0050b8ed None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:29.220261 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 419/846] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:29.227431 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e378d4-f142-47a2-9d88-3c416457f38d None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:29.228039 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e378d4-f142-47a2-9d88-3c416457f38d None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:29.228533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e378d4-f142-47a2-9d88-3c416457f38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:29.256023 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-77e378d4-f142-47a2-9d88-3c416457f38d None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:29.256754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-77e378d4-f142-47a2-9d88-3c416457f38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:29.348101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 428/847] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:00:28 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 20:00:29.350264 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 420/848] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:29.353632 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941713e-c724-4d9c-b552-310d0050b8ed None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:29.393765 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:29.467297 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941713e-c724-4d9c-b552-310d0050b8ed None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:29.504104 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25b3a6-99b9-4284-a366-785460b7cca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:29.504753 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25b3a6-99b9-4284-a366-785460b7cca3 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:29.505033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25b3a6-99b9-4284-a366-785460b7cca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:29.505307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25b3a6-99b9-4284-a366-785460b7cca3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:29.516342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25b3a6-99b9-4284-a366-785460b7cca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:29.517822 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25b3a6-99b9-4284-a366-785460b7cca3 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:29.533793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-4d25b3a6-99b9-4284-a366-785460b7cca3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:29.533793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:29.533793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:29.533793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:29.533793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:29.533793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:29.533793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:29.533793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:29.533793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:29.533793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:29.533793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:29.533793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:29.533793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:29.533793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:29.533793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:29.533793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:29.534782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:29.534782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:29.534782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:29.534782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:29.534782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:29.534782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:29.534782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:29.534782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:29.534782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:29.534782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:29.534782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:29.534782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:29.534782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:29.534782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:29.534782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:29.534782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:29.534782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:29.535675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:29.535675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:29.535675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:29.535675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 429/849] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:29.540846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e378d4-f142-47a2-9d88-3c416457f38d None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:29.543621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:29.602063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c71dc8-f880-4353-b75f-4b2f31090e0e None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:29.602412 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c71dc8-f880-4353-b75f-4b2f31090e0e None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:29.602610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c71dc8-f880-4353-b75f-4b2f31090e0e None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:29.605255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 430/850] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:29.609528 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee524a2-2165-488c-865d-003cebc2c104 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:29.609851 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee524a2-2165-488c-865d-003cebc2c104 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:29.610124 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee524a2-2165-488c-865d-003cebc2c104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:29.632196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-cee524a2-2165-488c-865d-003cebc2c104 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:29.633406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-cee524a2-2165-488c-865d-003cebc2c104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:29.658809 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e378d4-f142-47a2-9d88-3c416457f38d None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:29.696563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa601b56-1b7f-446f-9797-01b52b5ed37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:29.697173 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa601b56-1b7f-446f-9797-01b52b5ed37e None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:29.697487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa601b56-1b7f-446f-9797-01b52b5ed37e None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:29.697834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa601b56-1b7f-446f-9797-01b52b5ed37e None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:29.698391 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa601b56-1b7f-446f-9797-01b52b5ed37e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:29.698692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa601b56-1b7f-446f-9797-01b52b5ed37e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:29.700053 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa601b56-1b7f-446f-9797-01b52b5ed37e None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:29.716048 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 421/852] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:29.716048 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa601b56-1b7f-446f-9797-01b52b5ed37e None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:29.716342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 431/851] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:00:29.718283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee524a2-2165-488c-865d-003cebc2c104 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:29.739525 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:29.784670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:29.828291 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa601b56-1b7f-446f-9797-01b52b5ed37e None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:29.834334 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee524a2-2165-488c-865d-003cebc2c104 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:29.866123 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b26394-b3dc-4f06-906c-2c4bffb111cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:29.866940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b26394-b3dc-4f06-906c-2c4bffb111cb None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:29.867156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b26394-b3dc-4f06-906c-2c4bffb111cb None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:29.867394 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b26394-b3dc-4f06-906c-2c4bffb111cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:29.876491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58425e-5286-42d5-a573-f7b75bc6e009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:29.877436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58425e-5286-42d5-a573-f7b75bc6e009 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:29.877983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58425e-5286-42d5-a573-f7b75bc6e009 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:29.878463 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58425e-5286-42d5-a573-f7b75bc6e009 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:29.878994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b26394-b3dc-4f06-906c-2c4bffb111cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:00:29.884501 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-a0b26394-b3dc-4f06-906c-2c4bffb111cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:00:29.884501 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:29.884501 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:29.884501 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:29.884501 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:29.884501 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:29.884501 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:29.884501 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:29.884501 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:29.884501 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:29.884501 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:29.884501 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:29.884501 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:00:29.884501 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:00:29.884501 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:00:29.884501 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:00:29.885527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:00:29.885527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:00:29.885527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:00:29.885527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:00:29.885527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:29.885527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:29.885527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:29.885527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:29.885527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:29.885527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:29.885527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:29.885527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:29.885527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:29.885527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:29.885527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:29.885527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:29.886739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:29.886739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:29.886739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:29.886739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:29.886739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:00:29.886739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:00:29.886739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:00:29.886739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:00:29.886739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:00:29.886739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:00:29.886739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:00:29.886739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:29.890664 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d58425e-5286-42d5-a573-f7b75bc6e009 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:29.892081 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 422/853] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 20:00:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:29.892942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d58425e-5286-42d5-a573-f7b75bc6e009 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:29.893072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b26394-b3dc-4f06-906c-2c4bffb111cb None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:29.906950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-5d58425e-5286-42d5-a573-f7b75bc6e009 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:29.906950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:29.906950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:29.906950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:29.906950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:29.906950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:29.906950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:29.906950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:29.906950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:29.906950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:29.906950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:29.906950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:29.906950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:29.906950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:29.906950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:29.907950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:29.907950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:29.907950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:29.907950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:29.907950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:29.907950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:29.907950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:29.907950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:29.907950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:29.907950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:29.907950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:29.907950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:29.907950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:29.907950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:29.907950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:29.907950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:29.907950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:29.908828 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:29.908828 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:29.908828 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:29.908828 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:29.908828 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:29.908828 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:29.908828 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:29.908828 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:29.915876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 432/854] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:29.917094 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58425e-5286-42d5-a573-f7b75bc6e009 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:30.001697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:30.004883 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b26394-b3dc-4f06-906c-2c4bffb111cb None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:30.027656 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58425e-5286-42d5-a573-f7b75bc6e009 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:30.030336 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:30.043579 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ebbc0-180e-47ac-b9a3-c6014b9a4dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:30.044243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ebbc0-180e-47ac-b9a3-c6014b9a4dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:30.044530 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ebbc0-180e-47ac-b9a3-c6014b9a4dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:30.044820 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ebbc0-180e-47ac-b9a3-c6014b9a4dd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:30.046120 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ebbc0-180e-47ac-b9a3-c6014b9a4dd2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:30.046425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ebbc0-180e-47ac-b9a3-c6014b9a4dd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:30.047860 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ebbc0-180e-47ac-b9a3-c6014b9a4dd2 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:30.061052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 423/855] 10.222.0.73 () {56 vars in 1103 bytes} [Mon Aug 29 20:00:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:30.061979 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ebbc0-180e-47ac-b9a3-c6014b9a4dd2 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:30.066935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503bbf5d-0412-4088-afbc-d97a5430732f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:30.067555 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503bbf5d-0412-4088-afbc-d97a5430732f None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:30.067835 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503bbf5d-0412-4088-afbc-d97a5430732f None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:30.068108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503bbf5d-0412-4088-afbc-d97a5430732f None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:30.068799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503bbf5d-0412-4088-afbc-d97a5430732f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:30.069106 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503bbf5d-0412-4088-afbc-d97a5430732f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:30.070568 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503bbf5d-0412-4088-afbc-d97a5430732f None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:30.085150 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 433/856] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:00:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:30.085150 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503bbf5d-0412-4088-afbc-d97a5430732f None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:30.189852 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ebbc0-180e-47ac-b9a3-c6014b9a4dd2 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:30.199015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503bbf5d-0412-4088-afbc-d97a5430732f None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:30.227292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07a002-dfbb-49b7-939b-a156305ff144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:30.227930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07a002-dfbb-49b7-939b-a156305ff144 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:30.228211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07a002-dfbb-49b7-939b-a156305ff144 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:30.228489 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07a002-dfbb-49b7-939b-a156305ff144 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:30.229072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07a002-dfbb-49b7-939b-a156305ff144 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:30.230511 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07a002-dfbb-49b7-939b-a156305ff144 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:30.242019 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda76480-5dca-45a8-8dd4-329f9094781d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:30.242709 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda76480-5dca-45a8-8dd4-329f9094781d None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:30.242989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda76480-5dca-45a8-8dd4-329f9094781d None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:30.243253 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda76480-5dca-45a8-8dd4-329f9094781d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:30.243357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-0c07a002-dfbb-49b7-939b-a156305ff144 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:00:30.243357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:30.243357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:00:30.243357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:00:30.243357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:00:30.243357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:00:30.243357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:00:30.243357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:30.243357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:00:30.243357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:30.243357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:30.243357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:30.243357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:30.243357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:30.243357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:30.243357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:30.244210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:30.244210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:30.244210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:30.244210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:30.244210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:30.244210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:30.244210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:30.244210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:00:30.244210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:00:30.244210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:00:30.244210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:00:30.244210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:30.244210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:30.244210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:30.244210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:30.244210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:30.244210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:30.245136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:30.245136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:30.245136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:30.245136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:30.245136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:30.245136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:30.245136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:30.245136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:30.245136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:30.245136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:30.245136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:00:30.245136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:30.245136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:00:30.245136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:30.245136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 424/857] 10.222.0.73 () {56 vars in 1132 bytes} [Mon Aug 29 20:00:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:30.245968 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07a002-dfbb-49b7-939b-a156305ff144 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:30.254344 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:30.266272 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda76480-5dca-45a8-8dd4-329f9094781d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:30.267901 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda76480-5dca-45a8-8dd4-329f9094781d None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:30.284240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-cda76480-5dca-45a8-8dd4-329f9094781d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:30.284240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:30.284240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:30.284240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:30.284240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:30.284240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:30.284240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:30.284240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:30.284240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:30.284240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:30.284240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:30.284240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:30.284240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:30.284240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:30.284240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:30.284240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:30.285445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:30.285445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:30.285445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:30.285445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:30.285445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:30.285445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:30.285445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:30.285445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:30.285445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:30.285445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:30.285445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:30.285445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:30.285445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:30.285445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:30.285445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:30.285445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:30.285445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:30.286366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:30.286366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:30.286366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:30.286366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 434/858] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:30.286601 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda76480-5dca-45a8-8dd4-329f9094781d None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:30.309950 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:30.365496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07a002-dfbb-49b7-939b-a156305ff144 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:30.396769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda76480-5dca-45a8-8dd4-329f9094781d None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:30.401555 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e9e69-e2ad-4459-855d-47be4e2b8dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:30.402201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e9e69-e2ad-4459-855d-47be4e2b8dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:30.402476 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e9e69-e2ad-4459-855d-47be4e2b8dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:30.402746 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e9e69-e2ad-4459-855d-47be4e2b8dd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:30.403535 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e9e69-e2ad-4459-855d-47be4e2b8dd9 None admin] RBAC: Adding query filter params (role.id=296ee2c194824485af71618619f82e09, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=b92e30da8f394f3abc19b9e86ef126e5) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:30.403845 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e9e69-e2ad-4459-855d-47be4e2b8dd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:30.405474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e9e69-e2ad-4459-855d-47be4e2b8dd9 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:30.411759 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 425/859] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:00:30 2022] GET /identity/v3/role_assignments?user.id=b92e30da8f394f3abc19b9e86ef126e5&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=296ee2c194824485af71618619f82e09 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:30.412549 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e9e69-e2ad-4459-855d-47be4e2b8dd9 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:30.443236 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fe630-29c9-4a27-b485-cb9eadd7a69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:30.443883 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fe630-29c9-4a27-b485-cb9eadd7a69d None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:30.444160 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fe630-29c9-4a27-b485-cb9eadd7a69d None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:30.444710 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fe630-29c9-4a27-b485-cb9eadd7a69d None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:30.445378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fe630-29c9-4a27-b485-cb9eadd7a69d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:30.445826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fe630-29c9-4a27-b485-cb9eadd7a69d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:30.448089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fe630-29c9-4a27-b485-cb9eadd7a69d None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:30.457899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 435/860] 10.222.0.73 () {56 vars in 1126 bytes} [Mon Aug 29 20:00:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:30.461747 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fe630-29c9-4a27-b485-cb9eadd7a69d None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:30.542514 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:30.558752 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e9e69-e2ad-4459-855d-47be4e2b8dd9 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:30.578462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:30.607429 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cadb2-7d9c-449b-a7b3-89c5eea5df6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:30.608297 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cadb2-7d9c-449b-a7b3-89c5eea5df6c None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:30.608689 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cadb2-7d9c-449b-a7b3-89c5eea5df6c None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:30.608964 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cadb2-7d9c-449b-a7b3-89c5eea5df6c None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:30.609150 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fe630-29c9-4a27-b485-cb9eadd7a69d None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:30.609613 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2cadb2-7d9c-449b-a7b3-89c5eea5df6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:30.609950 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2cadb2-7d9c-449b-a7b3-89c5eea5df6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:30.611305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2cadb2-7d9c-449b-a7b3-89c5eea5df6c None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:30.626062 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 426/861] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:30.631299 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cadb2-7d9c-449b-a7b3-89c5eea5df6c None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:30.658502 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cbaaea-db40-47d2-959a-a8349da36a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:30.659465 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbaaea-db40-47d2-959a-a8349da36a79 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:30.659890 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbaaea-db40-47d2-959a-a8349da36a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:30.660311 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbaaea-db40-47d2-959a-a8349da36a79 None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:30.661651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cbaaea-db40-47d2-959a-a8349da36a79 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:30.662185 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cbaaea-db40-47d2-959a-a8349da36a79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:30.664209 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cbaaea-db40-47d2-959a-a8349da36a79 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:30.671919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 436/862] 10.222.0.73 () {56 vars in 1126 bytes} [Mon Aug 29 20:00:30 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:30.757031 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cadb2-7d9c-449b-a7b3-89c5eea5df6c None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:30.798473 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa061f-172b-43f9-8444-bcbd775912f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:30.799382 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa061f-172b-43f9-8444-bcbd775912f3 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:30.799814 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa061f-172b-43f9-8444-bcbd775912f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:30.800233 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa061f-172b-43f9-8444-bcbd775912f3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:30.809084 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa061f-172b-43f9-8444-bcbd775912f3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:30.811553 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa061f-172b-43f9-8444-bcbd775912f3 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:30.825231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-abfa061f-172b-43f9-8444-bcbd775912f3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:00:30.825231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:30.825231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:30.825231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:30.825231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:30.825231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:30.825231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:30.825231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:30.825231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:30.825231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:30.825231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:30.825231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:30.825231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:00:30.825231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:00:30.825231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:00:30.825231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:00:30.827197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:30.827197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:30.827197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:30.827197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:30.827197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:30.827197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:30.827197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:30.827197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:30.827197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:30.827197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:30.827197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:30.827197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:30.827197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:30.827197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:30.827197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:30.827197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:30.827197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:30.827197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:30.828358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:30.828358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:30.828358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:00:30.828358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:00:30.828358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:00:30.828358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:00:30.828358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:00:30.828358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:00:30.828358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:00:30.828358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:30.833871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 427/863] 10.222.0.73 () {56 vars in 1117 bytes} [Mon Aug 29 20:00:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:30.836659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cbaaea-db40-47d2-959a-a8349da36a79 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:30.946495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cbaaea-db40-47d2-959a-a8349da36a79 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:30.964580 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:30.986222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311c147-94f0-41a2-bf03-451a64baceb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:30.986953 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311c147-94f0-41a2-bf03-451a64baceb7 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:30.987357 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311c147-94f0-41a2-bf03-451a64baceb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:30.987699 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311c147-94f0-41a2-bf03-451a64baceb7 None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:30.988509 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311c147-94f0-41a2-bf03-451a64baceb7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:30.988886 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311c147-94f0-41a2-bf03-451a64baceb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:30.990544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311c147-94f0-41a2-bf03-451a64baceb7 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:31.015091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 437/864] 10.222.0.73 () {56 vars in 1149 bytes} [Mon Aug 29 20:00:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:31.020884 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa061f-172b-43f9-8444-bcbd775912f3 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:31.056614 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:31.132102 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa061f-172b-43f9-8444-bcbd775912f3 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:31.152303 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832131f8-1320-4652-89c9-160b98f4e8a6 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:31.152589 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832131f8-1320-4652-89c9-160b98f4e8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:31.152872 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832131f8-1320-4652-89c9-160b98f4e8a6 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:31.155066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 438/865] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:00:31.160348 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0016c-d54c-40b1-837a-87cb43f11c71 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:31.160636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0016c-d54c-40b1-837a-87cb43f11c71 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:31.160918 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0016c-d54c-40b1-837a-87cb43f11c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:31.172281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa655e-2f02-4348-bfd4-9c017270510a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:31.173176 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa655e-2f02-4348-bfd4-9c017270510a None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:31.173604 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa655e-2f02-4348-bfd4-9c017270510a None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:31.174048 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa655e-2f02-4348-bfd4-9c017270510a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:31.180965 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-83d0016c-d54c-40b1-837a-87cb43f11c71 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:31.182213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-83d0016c-d54c-40b1-837a-87cb43f11c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:31.245854 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa655e-2f02-4348-bfd4-9c017270510a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:31.247534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa655e-2f02-4348-bfd4-9c017270510a None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:31.266086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 439/866] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:31 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:31.289995 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:31.356066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 428/867] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:00:31 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:00:31.358210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0016c-d54c-40b1-837a-87cb43f11c71 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:31.462592 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0016c-d54c-40b1-837a-87cb43f11c71 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:31.465225 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:31.499416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6b43d-4de3-42c2-9427-b2b27fda60d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:31.500051 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6b43d-4de3-42c2-9427-b2b27fda60d6 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:31.500579 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6b43d-4de3-42c2-9427-b2b27fda60d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:31.500851 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6b43d-4de3-42c2-9427-b2b27fda60d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:31.511287 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6b43d-4de3-42c2-9427-b2b27fda60d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:31.512743 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6b43d-4de3-42c2-9427-b2b27fda60d6 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:31.528326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-98e6b43d-4de3-42c2-9427-b2b27fda60d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:31.528326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:31.528326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:31.528326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:31.528326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:31.528326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:31.528326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:31.528326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:31.528326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:31.528326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:31.528326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:31.528326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:31.528326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:31.528326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:31.528326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:31.528326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:31.529909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:31.529909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:31.529909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:31.529909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:31.529909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:31.529909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:31.529909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:31.529909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:31.529909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:31.529909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:31.529909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:31.529909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:31.529909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:31.529909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:31.529909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:31.529909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:31.529909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:31.530840 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:31.530840 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:31.530840 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:31.533836 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 440/868] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:31.535998 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa655e-2f02-4348-bfd4-9c017270510a None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:31.552730 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6b43d-4de3-42c2-9427-b2b27fda60d6 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:31.643056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa655e-2f02-4348-bfd4-9c017270510a None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:31.668001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6b43d-4de3-42c2-9427-b2b27fda60d6 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:31.674771 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:31.680421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3f852-08d5-4718-bcdb-fd496e79b049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:31.681039 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3f852-08d5-4718-bcdb-fd496e79b049 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:31.681319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3f852-08d5-4718-bcdb-fd496e79b049 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:31.681589 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3f852-08d5-4718-bcdb-fd496e79b049 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:31.682264 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3f852-08d5-4718-bcdb-fd496e79b049 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:31.682569 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3f852-08d5-4718-bcdb-fd496e79b049 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:31.683931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3f852-08d5-4718-bcdb-fd496e79b049 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:31.703374 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 429/869] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:31.704741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3f852-08d5-4718-bcdb-fd496e79b049 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:31.712741 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a95e3-28af-4397-bdbf-ac7ffedd73b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:31.713571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a95e3-28af-4397-bdbf-ac7ffedd73b1 None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:31.714051 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a95e3-28af-4397-bdbf-ac7ffedd73b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:31.714935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a95e3-28af-4397-bdbf-ac7ffedd73b1 None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:31.715767 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a95e3-28af-4397-bdbf-ac7ffedd73b1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:31.718335 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a95e3-28af-4397-bdbf-ac7ffedd73b1 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:31.740612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 441/870] 10.222.0.73 () {60 vars in 1141 bytes} [Mon Aug 29 20:00:31 2022] POST /identity/v3/services => generated 251 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:00:31.831154 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3f852-08d5-4718-bcdb-fd496e79b049 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:31.880433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c269b4-0db5-4b0e-ab13-02d6ff86be64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:31.881074 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c269b4-0db5-4b0e-ab13-02d6ff86be64 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:31.881420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c269b4-0db5-4b0e-ab13-02d6ff86be64 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:31.881866 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c269b4-0db5-4b0e-ab13-02d6ff86be64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:31.890521 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c269b4-0db5-4b0e-ab13-02d6ff86be64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:31.892251 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c269b4-0db5-4b0e-ab13-02d6ff86be64 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:31.905834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-e7c269b4-0db5-4b0e-ab13-02d6ff86be64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:31.905834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:31.905834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:31.905834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:31.905834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:31.905834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:31.905834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:31.905834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:31.905834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:31.905834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:31.905834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:31.905834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:31.905834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:31.905834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:31.905834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:31.906731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:31.906731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:31.906731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:31.906731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:31.906731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:31.906731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:31.906731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:31.906731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:31.906731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:31.906731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:31.906731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:31.906731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:31.906731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:31.906731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:31.906731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:31.906731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:31.906731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:31.908133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:31.908133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:31.908133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:31.908133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:31.908133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:31.908133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:31.908133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:31.908133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:31.909834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 430/871] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:31.913620 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a95e3-28af-4397-bdbf-ac7ffedd73b1 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:31.934042 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:32.027451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a95e3-28af-4397-bdbf-ac7ffedd73b1 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:32.066285 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db438dfe-2b59-4639-9ff7-64f4a96622df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:32.066903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db438dfe-2b59-4639-9ff7-64f4a96622df None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:32.067182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db438dfe-2b59-4639-9ff7-64f4a96622df None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:32.067448 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db438dfe-2b59-4639-9ff7-64f4a96622df None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:32.068142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db438dfe-2b59-4639-9ff7-64f4a96622df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:32.068447 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db438dfe-2b59-4639-9ff7-64f4a96622df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:32.069904 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db438dfe-2b59-4639-9ff7-64f4a96622df None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:32.083776 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 442/872] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:00:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:32.090794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c269b4-0db5-4b0e-ab13-02d6ff86be64 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:32.149061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:32.204703 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c269b4-0db5-4b0e-ab13-02d6ff86be64 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:32.240978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f7393-c933-4233-8859-f140a90c8cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:32.241594 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f7393-c933-4233-8859-f140a90c8cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:32.241948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f7393-c933-4233-8859-f140a90c8cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:32.242227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f7393-c933-4233-8859-f140a90c8cd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:32.242956 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f7393-c933-4233-8859-f140a90c8cd2 None admin] RBAC: Adding query filter params (role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=e0c76478b6694782b896cdf9cda32d9c) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:32.243306 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f7393-c933-4233-8859-f140a90c8cd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:32.244737 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f7393-c933-4233-8859-f140a90c8cd2 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:32.252328 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 431/873] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:00:32 2022] GET /identity/v3/role_assignments?user.id=e0c76478b6694782b896cdf9cda32d9c&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:32.506161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407806e-06ca-4fd3-a1ae-93ced2dff135 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:32.506466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407806e-06ca-4fd3-a1ae-93ced2dff135 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:32.506665 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407806e-06ca-4fd3-a1ae-93ced2dff135 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:32.508238 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 443/874] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:32.513667 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf9381-ff3a-4869-a272-51bf1193ad7b None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:32.514499 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf9381-ff3a-4869-a272-51bf1193ad7b None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:32.514776 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf9381-ff3a-4869-a272-51bf1193ad7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:32.538621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-eacf9381-ff3a-4869-a272-51bf1193ad7b None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:32.540142 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-eacf9381-ff3a-4869-a272-51bf1193ad7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:32.618012 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 432/875] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:32 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:32.833918 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e887053-ab28-43d9-957d-cb690a2093bc None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:32.834697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e887053-ab28-43d9-957d-cb690a2093bc None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:32.835200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e887053-ab28-43d9-957d-cb690a2093bc None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:32.837071 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 444/876] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:32.843175 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad03344-0f88-423d-8f45-661d9b5b3b63 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:32.843461 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad03344-0f88-423d-8f45-661d9b5b3b63 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:32.843735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad03344-0f88-423d-8f45-661d9b5b3b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:32.863805 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-0ad03344-0f88-423d-8f45-661d9b5b3b63 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:32.864998 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-0ad03344-0f88-423d-8f45-661d9b5b3b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:32.891165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e887053-ab28-43d9-957d-cb690a2093bc None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:32.903811 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:32.945841 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 433/877] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:32 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:00:32.984261 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:33.012411 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e887053-ab28-43d9-957d-cb690a2093bc None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:33.050075 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecac497-1298-42a1-a1b4-8c6264bf1ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:33.051070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecac497-1298-42a1-a1b4-8c6264bf1ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:33.051557 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecac497-1298-42a1-a1b4-8c6264bf1ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:33.052052 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecac497-1298-42a1-a1b4-8c6264bf1ac5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:33.052850 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecac497-1298-42a1-a1b4-8c6264bf1ac5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:33.054450 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecac497-1298-42a1-a1b4-8c6264bf1ac5 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:33.064708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-8ecac497-1298-42a1-a1b4-8c6264bf1ac5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:00:33.064708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:33.064708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:00:33.064708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:00:33.064708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:00:33.064708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:00:33.064708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:00:33.064708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:33.064708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:00:33.064708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:33.064708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:33.064708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:33.064708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:33.064708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:33.064708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:33.064708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:33.064708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:33.065775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:33.065775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:33.065775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:33.065775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:33.065775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:33.065775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:33.065775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:00:33.065775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:00:33.065775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:00:33.065775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:00:33.065775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:33.065775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:33.065775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:33.065775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:33.065775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:33.065775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:33.066624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:33.066624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:33.066624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:33.066624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:33.066624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:33.066624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:33.066624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:33.066624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:33.066624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:33.066624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:33.066624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:00:33.066624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:33.066624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:00:33.066624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:33.069468 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 445/878] 10.222.0.73 () {56 vars in 1126 bytes} [Mon Aug 29 20:00:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:33.075634 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad03344-0f88-423d-8f45-661d9b5b3b63 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:33.185333 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad03344-0f88-423d-8f45-661d9b5b3b63 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:33.226956 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd24d0-48de-448d-a475-38eb0c95645e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:33.227572 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd24d0-48de-448d-a475-38eb0c95645e None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:33.227850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd24d0-48de-448d-a475-38eb0c95645e None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:33.228125 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd24d0-48de-448d-a475-38eb0c95645e None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:33.228754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd24d0-48de-448d-a475-38eb0c95645e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:33.229055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd24d0-48de-448d-a475-38eb0c95645e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:33.230506 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd24d0-48de-448d-a475-38eb0c95645e None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:33.234533 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecac497-1298-42a1-a1b4-8c6264bf1ac5 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:33.235807 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 434/879] 10.222.0.73 () {56 vars in 1122 bytes} [Mon Aug 29 20:00:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:33.247275 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd24d0-48de-448d-a475-38eb0c95645e None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:33.294282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:33.366840 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecac497-1298-42a1-a1b4-8c6264bf1ac5 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:33.384406 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd24d0-48de-448d-a475-38eb0c95645e None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:33.413432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c539e5ce-c0fc-454c-99dc-2f8ea6fe5f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:33.414608 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539e5ce-c0fc-454c-99dc-2f8ea6fe5f51 None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:33.414954 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539e5ce-c0fc-454c-99dc-2f8ea6fe5f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:33.415293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539e5ce-c0fc-454c-99dc-2f8ea6fe5f51 None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:33.416311 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c539e5ce-c0fc-454c-99dc-2f8ea6fe5f51 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:33.418576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c539e5ce-c0fc-454c-99dc-2f8ea6fe5f51 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:33.430278 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a98a49-93d9-46ff-8129-5352d5fb6ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:33.431334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a98a49-93d9-46ff-8129-5352d5fb6ade None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:33.431948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a98a49-93d9-46ff-8129-5352d5fb6ade None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:33.432724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:33.433768 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a98a49-93d9-46ff-8129-5352d5fb6ade None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:33.434829 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a98a49-93d9-46ff-8129-5352d5fb6ade None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:33.435892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a98a49-93d9-46ff-8129-5352d5fb6ade None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:33.438886 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a98a49-93d9-46ff-8129-5352d5fb6ade None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:33.441877 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 446/880] 10.222.0.73 () {60 vars in 1141 bytes} [Mon Aug 29 20:00:33 2022] POST /identity/v3/services => generated 269 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:00:33.458128 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 435/881] 10.222.0.73 () {56 vars in 1122 bytes} [Mon Aug 29 20:00:33 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:33.836906 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af19a52-53ca-42ff-9b9c-edbb6760426b None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:33.837285 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af19a52-53ca-42ff-9b9c-edbb6760426b None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:33.837484 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af19a52-53ca-42ff-9b9c-edbb6760426b None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:33.839110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 447/882] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:33.845218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df959bd-85bc-4950-82d4-95bd02961747 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:33.845508 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df959bd-85bc-4950-82d4-95bd02961747 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:33.845813 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df959bd-85bc-4950-82d4-95bd02961747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:33.865553 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-0df959bd-85bc-4950-82d4-95bd02961747 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:33.866794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-0df959bd-85bc-4950-82d4-95bd02961747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:33.947591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 436/883] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:34.223083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af19a52-53ca-42ff-9b9c-edbb6760426b None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:34.268479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8f6fa-46fe-4131-95e1-64fd50449af9 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:34.269032 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8f6fa-46fe-4131-95e1-64fd50449af9 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:34.269032 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8f6fa-46fe-4131-95e1-64fd50449af9 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:34.270378 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 437/884] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:34.276718 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31674e3-5413-4be8-ba25-3f5dd77841c6 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:34.277017 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31674e3-5413-4be8-ba25-3f5dd77841c6 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:34.277293 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31674e3-5413-4be8-ba25-3f5dd77841c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:34.296147 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-c31674e3-5413-4be8-ba25-3f5dd77841c6 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:34.297352 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c31674e3-5413-4be8-ba25-3f5dd77841c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:34.318132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:34.335892 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af19a52-53ca-42ff-9b9c-edbb6760426b None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:34.373825 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 438/885] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:00:34.377725 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31395588-124f-4d14-a33f-9a593fc5fd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:34.378486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31395588-124f-4d14-a33f-9a593fc5fd06 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:34.378768 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31395588-124f-4d14-a33f-9a593fc5fd06 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:34.379091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31395588-124f-4d14-a33f-9a593fc5fd06 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:34.379686 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31395588-124f-4d14-a33f-9a593fc5fd06 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:34.381133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31395588-124f-4d14-a33f-9a593fc5fd06 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:34.392340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-31395588-124f-4d14-a33f-9a593fc5fd06 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:00:34.392340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:34.392340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:00:34.392340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:00:34.392340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:00:34.392340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:00:34.392340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:00:34.392340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:34.392340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:00:34.392340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:34.392340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:34.392340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:34.392340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:34.392340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:34.392340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:34.392340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:34.392340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:34.393384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:34.393384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:34.393384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:34.393384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:34.393384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:34.393384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:34.393384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:00:34.393384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:00:34.393384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:00:34.393384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:00:34.393384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:34.393384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:34.393384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:34.393384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:34.393384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:34.393384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:34.394247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:34.394247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:34.394247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:34.394247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:34.394247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:34.394247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:34.394247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:34.394247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:34.394247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:34.394247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:34.394247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:00:34.394247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:34.394247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:00:34.394247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:34.394247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 448/886] 10.222.0.73 () {56 vars in 1114 bytes} [Mon Aug 29 20:00:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:34.399274 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31674e3-5413-4be8-ba25-3f5dd77841c6 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:34.510861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31674e3-5413-4be8-ba25-3f5dd77841c6 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:34.547839 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9497dd-6ece-4493-b34d-e36574248c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:34.548452 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9497dd-6ece-4493-b34d-e36574248c34 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:34.548732 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9497dd-6ece-4493-b34d-e36574248c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:34.549004 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9497dd-6ece-4493-b34d-e36574248c34 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:34.549636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9497dd-6ece-4493-b34d-e36574248c34 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:34.549980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9497dd-6ece-4493-b34d-e36574248c34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:34.551345 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9497dd-6ece-4493-b34d-e36574248c34 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:34.556501 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 439/887] 10.222.0.73 () {56 vars in 1114 bytes} [Mon Aug 29 20:00:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:34.562060 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31395588-124f-4d14-a33f-9a593fc5fd06 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:34.638288 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:34.679203 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9497dd-6ece-4493-b34d-e36574248c34 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:34.679899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31395588-124f-4d14-a33f-9a593fc5fd06 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:34.705370 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:34.720200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86a0e6-57bc-4f9e-90c8-281f1a85ae2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:34.721132 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86a0e6-57bc-4f9e-90c8-281f1a85ae2c None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:34.721411 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86a0e6-57bc-4f9e-90c8-281f1a85ae2c None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:34.721925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86a0e6-57bc-4f9e-90c8-281f1a85ae2c None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:34.722666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86a0e6-57bc-4f9e-90c8-281f1a85ae2c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:34.722977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86a0e6-57bc-4f9e-90c8-281f1a85ae2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:34.724343 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86a0e6-57bc-4f9e-90c8-281f1a85ae2c None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:34.735491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 449/888] 10.222.0.73 () {56 vars in 1114 bytes} [Mon Aug 29 20:00:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:34.737093 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86a0e6-57bc-4f9e-90c8-281f1a85ae2c None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:34.801581 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9497dd-6ece-4493-b34d-e36574248c34 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:34.838296 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e484b31-3458-474e-9da9-9cdeed28c186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:34.838928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e484b31-3458-474e-9da9-9cdeed28c186 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:34.839203 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e484b31-3458-474e-9da9-9cdeed28c186 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:34.839589 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e484b31-3458-474e-9da9-9cdeed28c186 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:34.852994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e484b31-3458-474e-9da9-9cdeed28c186 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:00:34.855225 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86a0e6-57bc-4f9e-90c8-281f1a85ae2c None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:34.858816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-4e484b31-3458-474e-9da9-9cdeed28c186 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:00:34.858816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:34.858816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:34.858816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:34.858816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:34.858816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:34.858816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:34.858816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:34.858816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:34.858816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:34.858816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:34.858816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:34.858816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:00:34.858816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:00:34.858816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:00:34.858816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:00:34.859766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:00:34.859766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:00:34.859766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:00:34.859766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:00:34.859766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:34.859766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:34.859766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:34.859766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:34.859766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:34.859766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:34.859766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:34.859766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:34.859766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:34.859766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:34.859766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:34.859766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:34.860871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:34.860871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:34.860871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:34.860871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:34.860871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:00:34.860871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:00:34.860871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:00:34.860871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:00:34.860871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:00:34.860871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:00:34.860871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:00:34.860871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:34.860871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 440/889] 10.222.0.73 () {56 vars in 1120 bytes} [Mon Aug 29 20:00:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:00:34.867862 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e484b31-3458-474e-9da9-9cdeed28c186 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:34.894215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:34.910560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89930c-5469-4956-9eb4-88e53bbc9fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:34.911114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89930c-5469-4956-9eb4-88e53bbc9fdb None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:34.911390 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89930c-5469-4956-9eb4-88e53bbc9fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:34.911658 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89930c-5469-4956-9eb4-88e53bbc9fdb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:34.912354 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89930c-5469-4956-9eb4-88e53bbc9fdb None admin] RBAC: Adding query filter params (interface=public, service_id=1042bd0a60f7404fb618c26490fea321, region_id=RegionOne) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:34.912659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89930c-5469-4956-9eb4-88e53bbc9fdb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:34.914365 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89930c-5469-4956-9eb4-88e53bbc9fdb None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:34.928319 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 450/890] 10.222.0.73 () {56 vars in 1251 bytes} [Mon Aug 29 20:00:34 2022] GET /identity/v3/endpoints?service_id=1042bd0a60f7404fb618c26490fea321&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:34.929468 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89930c-5469-4956-9eb4-88e53bbc9fdb None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:34.994882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:35.022900 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e484b31-3458-474e-9da9-9cdeed28c186 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:35.056010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89930c-5469-4956-9eb4-88e53bbc9fdb None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:35.062821 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d7bba-411f-46c7-ab20-380a225b4fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:35.063432 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d7bba-411f-46c7-ab20-380a225b4fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:35.063707 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d7bba-411f-46c7-ab20-380a225b4fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:35.063976 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d7bba-411f-46c7-ab20-380a225b4fc8 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:35.064640 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d7bba-411f-46c7-ab20-380a225b4fc8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:35.065003 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d7bba-411f-46c7-ab20-380a225b4fc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:35.066395 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d7bba-411f-46c7-ab20-380a225b4fc8 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:35.081472 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 441/891] 10.222.0.73 () {56 vars in 1115 bytes} [Mon Aug 29 20:00:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:35.085567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d7bba-411f-46c7-ab20-380a225b4fc8 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:35.112894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15951459-092b-4ff3-9afa-5101fa60a4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:35.114462 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15951459-092b-4ff3-9afa-5101fa60a4ca None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:35.114949 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15951459-092b-4ff3-9afa-5101fa60a4ca None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:35.115747 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15951459-092b-4ff3-9afa-5101fa60a4ca None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:35.116847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15951459-092b-4ff3-9afa-5101fa60a4ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:35.119331 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15951459-092b-4ff3-9afa-5101fa60a4ca None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:35.129454 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 451/892] 10.222.0.73 () {56 vars in 1087 bytes} [Mon Aug 29 20:00:34 2022] GET /identity/v3/services => generated 1268 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:00:35.220401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d7bba-411f-46c7-ab20-380a225b4fc8 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:35.260946 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3ec35-cb84-461d-9f26-86e1b4cee5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:35.261592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3ec35-cb84-461d-9f26-86e1b4cee5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:35.261909 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3ec35-cb84-461d-9f26-86e1b4cee5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:35.262184 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3ec35-cb84-461d-9f26-86e1b4cee5a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:35.274514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3ec35-cb84-461d-9f26-86e1b4cee5a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:35.275968 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3ec35-cb84-461d-9f26-86e1b4cee5a8 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:35.277912 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:35.292509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-59e3ec35-cb84-461d-9f26-86e1b4cee5a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:35.292509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:35.292509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:35.292509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:35.292509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:35.292509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:35.292509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:35.292509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:35.292509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:35.292509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:35.292509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:35.292509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:35.292509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:35.292509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:35.292509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:35.292509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:35.293623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:35.293623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:35.293623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:35.293623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:35.293623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:35.293623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:35.293623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:35.293623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:35.293623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:35.293623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:35.293623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:35.293623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:35.293623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:35.293623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:35.293623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:35.293623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:35.293623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:35.294533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:35.294533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:35.294533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:35.295140 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 442/893] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:35.299909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15951459-092b-4ff3-9afa-5101fa60a4ca None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:35.365189 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:35.415292 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15951459-092b-4ff3-9afa-5101fa60a4ca None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:35.451550 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bafc8f-1783-42f4-8f56-25d678c305e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:35.452541 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bafc8f-1783-42f4-8f56-25d678c305e3 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:35.453056 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bafc8f-1783-42f4-8f56-25d678c305e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:35.453534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bafc8f-1783-42f4-8f56-25d678c305e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:35.454415 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bafc8f-1783-42f4-8f56-25d678c305e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:35.454928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bafc8f-1783-42f4-8f56-25d678c305e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:35.456493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bafc8f-1783-42f4-8f56-25d678c305e3 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:35.467754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df79eb-5387-4d45-b819-f13b07733d66 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:35.468184 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df79eb-5387-4d45-b819-f13b07733d66 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:35.468539 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df79eb-5387-4d45-b819-f13b07733d66 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:35.470142 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 443/894] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:35.471423 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 452/895] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:35.475619 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcc987-12e6-475c-9b0d-ee6844fafa52 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:35.475908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcc987-12e6-475c-9b0d-ee6844fafa52 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:35.476182 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcc987-12e6-475c-9b0d-ee6844fafa52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:35.477020 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bafc8f-1783-42f4-8f56-25d678c305e3 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:35.496819 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-91dcc987-12e6-475c-9b0d-ee6844fafa52 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:35.498230 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-91dcc987-12e6-475c-9b0d-ee6844fafa52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:35.570807 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 444/896] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:35 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:35.579226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b830e-fa0a-439c-8f66-176c9e3e1678 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:35.579514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b830e-fa0a-439c-8f66-176c9e3e1678 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:35.579793 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b830e-fa0a-439c-8f66-176c9e3e1678 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:35.581279 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 445/897] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:35 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:35.586858 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56926f-ebce-48b5-945f-d14d526065c7 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:35.587202 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56926f-ebce-48b5-945f-d14d526065c7 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:35.587400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56926f-ebce-48b5-945f-d14d526065c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:35.589736 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bafc8f-1783-42f4-8f56-25d678c305e3 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:35.607618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-cb56926f-ebce-48b5-945f-d14d526065c7 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:35.608933 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:35.610107 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-cb56926f-ebce-48b5-945f-d14d526065c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:35.636529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3a134-9e10-4e3d-ad01-37a80e64940b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:35.637539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3a134-9e10-4e3d-ad01-37a80e64940b None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:35.637970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3a134-9e10-4e3d-ad01-37a80e64940b None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:35.638249 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3a134-9e10-4e3d-ad01-37a80e64940b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:35.649103 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3a134-9e10-4e3d-ad01-37a80e64940b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:35.651651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3a134-9e10-4e3d-ad01-37a80e64940b None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:35.666847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-c8e3a134-9e10-4e3d-ad01-37a80e64940b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:00:35.666847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:35.666847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:35.666847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:35.666847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:35.666847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:35.666847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:35.666847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:35.666847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:35.666847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:35.666847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:35.666847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:35.666847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:00:35.666847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:00:35.666847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:00:35.666847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:00:35.667862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:35.667862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:35.667862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:35.667862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:35.667862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:35.667862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:35.667862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:35.667862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:35.667862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:35.667862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:35.667862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:35.667862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:35.667862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:35.667862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:35.667862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:35.667862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:35.667862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:35.667862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:35.669097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:35.669097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:35.669097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:00:35.669097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:00:35.669097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:00:35.669097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:00:35.669097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:00:35.669097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:00:35.669097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:00:35.669097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:35.669097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 453/898] 10.222.0.73 () {56 vars in 1117 bytes} [Mon Aug 29 20:00:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:00:35.675011 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3a134-9e10-4e3d-ad01-37a80e64940b None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:35.692728 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 446/899] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:35 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:35.785102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3a134-9e10-4e3d-ad01-37a80e64940b None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:35.820719 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033632d-0d12-493b-b667-20f6449c1152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:35.821805 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033632d-0d12-493b-b667-20f6449c1152 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:35.822322 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033632d-0d12-493b-b667-20f6449c1152 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:35.822784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033632d-0d12-493b-b667-20f6449c1152 None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:35.823698 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d033632d-0d12-493b-b667-20f6449c1152 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:35.824195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d033632d-0d12-493b-b667-20f6449c1152 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:35.825949 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d033632d-0d12-493b-b667-20f6449c1152 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:35.838670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56926f-ebce-48b5-945f-d14d526065c7 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:35.845972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 454/900] 10.222.0.73 () {56 vars in 1149 bytes} [Mon Aug 29 20:00:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:35.852981 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033632d-0d12-493b-b667-20f6449c1152 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:35.893399 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:35.929415 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:35.970306 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033632d-0d12-493b-b667-20f6449c1152 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:35.989197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56926f-ebce-48b5-945f-d14d526065c7 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:36.006660 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d980d34-d0c2-428c-afdf-74df48111934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:36.007734 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d980d34-d0c2-428c-afdf-74df48111934 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:36.008484 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d980d34-d0c2-428c-afdf-74df48111934 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:36.008993 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d980d34-d0c2-428c-afdf-74df48111934 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:36.025074 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de82b7c-6095-4459-b24e-696a534f5f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:36.025681 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de82b7c-6095-4459-b24e-696a534f5f68 None admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:36.025981 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de82b7c-6095-4459-b24e-696a534f5f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:36.026242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de82b7c-6095-4459-b24e-696a534f5f68 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:36.026889 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de82b7c-6095-4459-b24e-696a534f5f68 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:36.028238 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de82b7c-6095-4459-b24e-696a534f5f68 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:36.044701 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 447/901] 10.222.0.73 () {60 vars in 1141 bytes} [Mon Aug 29 20:00:35 2022] POST /identity/v3/services => generated 241 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:00:36.046287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de82b7c-6095-4459-b24e-696a534f5f68 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:36.111372 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d980d34-d0c2-428c-afdf-74df48111934 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:36.113082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d980d34-d0c2-428c-afdf-74df48111934 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:36.164708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:36.198966 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de82b7c-6095-4459-b24e-696a534f5f68 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:36.211452 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:36.242020 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 455/902] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:00:35 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 20:00:36.243461 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095271c3-f789-49bb-bba7-bf1e28cb1479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:36.244271 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095271c3-f789-49bb-bba7-bf1e28cb1479 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:36.244823 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095271c3-f789-49bb-bba7-bf1e28cb1479 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:36.245241 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095271c3-f789-49bb-bba7-bf1e28cb1479 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:36.246001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095271c3-f789-49bb-bba7-bf1e28cb1479 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:36.247528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095271c3-f789-49bb-bba7-bf1e28cb1479 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:36.247706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d980d34-d0c2-428c-afdf-74df48111934 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:36.260621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-095271c3-f789-49bb-bba7-bf1e28cb1479 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:00:36.260621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:36.260621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:00:36.260621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:00:36.260621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:00:36.260621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:00:36.260621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:00:36.260621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:36.260621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:00:36.260621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:36.260621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:36.260621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:36.260621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:36.260621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:36.260621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:36.260621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:36.261832 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:36.261832 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:36.261832 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:36.261832 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:36.261832 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:36.261832 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:36.261832 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:36.261832 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:00:36.261832 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:00:36.261832 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:00:36.261832 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:00:36.261832 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:36.261832 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:36.261832 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:36.261832 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:36.261832 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:36.261832 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:36.262794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:36.262794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:36.262794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:36.262794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:36.262794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:36.262794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:36.262794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:36.262794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:36.262794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:36.262794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:36.262794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:00:36.262794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:36.262794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:00:36.262794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:36.265269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 448/903] 10.222.0.73 () {56 vars in 1132 bytes} [Mon Aug 29 20:00:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:36.271215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095271c3-f789-49bb-bba7-bf1e28cb1479 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:36.384863 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d980d34-d0c2-428c-afdf-74df48111934 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:36.385393 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095271c3-f789-49bb-bba7-bf1e28cb1479 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:36.422972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bf08a-d906-42d5-8a5c-d4bcba80419b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:36.423530 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bf08a-d906-42d5-8a5c-d4bcba80419b None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:36.423937 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bf08a-d906-42d5-8a5c-d4bcba80419b None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:36.424212 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bf08a-d906-42d5-8a5c-d4bcba80419b None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:36.424841 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bf08a-d906-42d5-8a5c-d4bcba80419b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:36.425148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bf08a-d906-42d5-8a5c-d4bcba80419b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:36.426559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bf08a-d906-42d5-8a5c-d4bcba80419b None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:36.429440 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c4700-a20d-4431-896a-403c4ff1cdd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:36.430482 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c4700-a20d-4431-896a-403c4ff1cdd8 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:36.430864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c4700-a20d-4431-896a-403c4ff1cdd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:36.431238 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c4700-a20d-4431-896a-403c4ff1cdd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:36.431902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 449/904] 10.222.0.73 () {56 vars in 1126 bytes} [Mon Aug 29 20:00:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:36.433385 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:36.437737 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bf08a-d906-42d5-8a5c-d4bcba80419b None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:36.456783 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c4700-a20d-4431-896a-403c4ff1cdd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:36.458772 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c4700-a20d-4431-896a-403c4ff1cdd8 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:36.478345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-d74c4700-a20d-4431-896a-403c4ff1cdd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:36.478345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:36.478345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:36.478345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:36.478345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:36.478345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:36.478345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:36.478345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:36.478345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:36.478345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:36.478345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:36.478345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:36.478345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:36.478345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:36.478345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:36.478345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:36.479364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:36.479364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:36.479364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:36.479364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:36.479364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:36.479364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:36.479364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:36.479364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:36.479364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:36.479364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:36.479364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:36.479364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:36.479364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:36.479364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:36.479364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:36.479364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:36.479364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:36.480900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:36.480900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:36.480900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:36.484334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:36.484884 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 456/905] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:00:36.486578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c4700-a20d-4431-896a-403c4ff1cdd8 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:36.556031 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bf08a-d906-42d5-8a5c-d4bcba80419b None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:36.594542 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1636abd-0575-4048-b2f6-87a0fee2ac34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:36.595541 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1636abd-0575-4048-b2f6-87a0fee2ac34 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:36.596026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1636abd-0575-4048-b2f6-87a0fee2ac34 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:36.596522 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1636abd-0575-4048-b2f6-87a0fee2ac34 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:36.597354 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1636abd-0575-4048-b2f6-87a0fee2ac34 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:36.597889 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1636abd-0575-4048-b2f6-87a0fee2ac34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:36.599543 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1636abd-0575-4048-b2f6-87a0fee2ac34 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:36.605512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 450/906] 10.222.0.73 () {56 vars in 1126 bytes} [Mon Aug 29 20:00:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:36.612133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1636abd-0575-4048-b2f6-87a0fee2ac34 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:36.628314 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c4700-a20d-4431-896a-403c4ff1cdd8 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:36.671814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc669e5a-8a9a-4a97-b97c-16e8a53bdc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:36.673068 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc669e5a-8a9a-4a97-b97c-16e8a53bdc0b None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:36.673397 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc669e5a-8a9a-4a97-b97c-16e8a53bdc0b None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:36.673808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc669e5a-8a9a-4a97-b97c-16e8a53bdc0b None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:36.674983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc669e5a-8a9a-4a97-b97c-16e8a53bdc0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:36.675390 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc669e5a-8a9a-4a97-b97c-16e8a53bdc0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:36.677312 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc669e5a-8a9a-4a97-b97c-16e8a53bdc0b None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:36.698878 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 457/907] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:36.700498 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc669e5a-8a9a-4a97-b97c-16e8a53bdc0b None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:36.738453 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1636abd-0575-4048-b2f6-87a0fee2ac34 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:36.740026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:36.757174 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:36.794233 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2a4c1-0d75-4cee-bdbc-9986431c2190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:36.794854 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2a4c1-0d75-4cee-bdbc-9986431c2190 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:36.795254 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2a4c1-0d75-4cee-bdbc-9986431c2190 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:36.796034 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2a4c1-0d75-4cee-bdbc-9986431c2190 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:36.797183 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2a4c1-0d75-4cee-bdbc-9986431c2190 None admin] RBAC: Adding query filter params (interface=public, service_id=ea786fea5449402b996c56917dc230b9, region_id=RegionOne) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:36.797501 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2a4c1-0d75-4cee-bdbc-9986431c2190 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:36.799216 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2a4c1-0d75-4cee-bdbc-9986431c2190 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:36.809873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 451/908] 10.222.0.73 () {56 vars in 1251 bytes} [Mon Aug 29 20:00:36 2022] GET /identity/v3/endpoints?service_id=ea786fea5449402b996c56917dc230b9&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:36.812849 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2a4c1-0d75-4cee-bdbc-9986431c2190 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:36.825277 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc669e5a-8a9a-4a97-b97c-16e8a53bdc0b None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:36.864926 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d67d6-71dc-4605-830f-bcecc5040bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:36.865563 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d67d6-71dc-4605-830f-bcecc5040bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:36.865887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d67d6-71dc-4605-830f-bcecc5040bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:36.866159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d67d6-71dc-4605-830f-bcecc5040bc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:36.873633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d67d6-71dc-4605-830f-bcecc5040bc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:36.875452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d67d6-71dc-4605-830f-bcecc5040bc7 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:36.887567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-a01d67d6-71dc-4605-830f-bcecc5040bc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:36.887567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:36.887567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:36.887567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:36.887567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:36.887567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:36.887567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:36.887567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:36.887567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:36.887567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:36.887567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:36.887567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:36.887567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:36.887567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:36.887567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:36.888495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:36.888495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:36.888495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:36.888495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:36.888495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:36.888495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:36.888495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:36.888495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:36.888495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:36.888495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:36.888495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:36.888495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:36.888495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:36.888495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:36.888495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:36.888495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:36.888495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:36.889596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:36.889596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:36.889596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:36.889596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:36.889596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:36.889596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:36.889596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:36.889596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:36.893777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 458/909] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:36.894804 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d67d6-71dc-4605-830f-bcecc5040bc7 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:36.953954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2a4c1-0d75-4cee-bdbc-9986431c2190 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:37.005105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3263df3f-3c2b-49a6-9dbe-70ce720f4c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:37.006355 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263df3f-3c2b-49a6-9dbe-70ce720f4c48 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:37.006840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263df3f-3c2b-49a6-9dbe-70ce720f4c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:37.007414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263df3f-3c2b-49a6-9dbe-70ce720f4c48 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:37.007703 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d67d6-71dc-4605-830f-bcecc5040bc7 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:37.008484 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3263df3f-3c2b-49a6-9dbe-70ce720f4c48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:37.011304 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3263df3f-3c2b-49a6-9dbe-70ce720f4c48 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:37.016236 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:37.026097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 452/910] 10.222.0.73 () {56 vars in 1087 bytes} [Mon Aug 29 20:00:36 2022] GET /identity/v3/services => generated 1497 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:00:37.055160 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae83081-34f0-437d-879d-7881e337624b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:37.055786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae83081-34f0-437d-879d-7881e337624b None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:37.056061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae83081-34f0-437d-879d-7881e337624b None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:37.056330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae83081-34f0-437d-879d-7881e337624b None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:37.057027 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae83081-34f0-437d-879d-7881e337624b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:37.057331 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae83081-34f0-437d-879d-7881e337624b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:37.058823 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae83081-34f0-437d-879d-7881e337624b None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:37.073108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 459/911] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:00:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:37.079271 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3263df3f-3c2b-49a6-9dbe-70ce720f4c48 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:37.117236 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:37.153225 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c43338-535c-4162-9b1d-e0b253e4cec7 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:37.153492 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c43338-535c-4162-9b1d-e0b253e4cec7 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:37.153860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c43338-535c-4162-9b1d-e0b253e4cec7 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:37.155483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 460/912] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:00:37.161193 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a8d0a-b3b1-4daa-b933-92bbd297d82a None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:37.161489 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a8d0a-b3b1-4daa-b933-92bbd297d82a None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:37.161765 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a8d0a-b3b1-4daa-b933-92bbd297d82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:37.184581 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-a95a8d0a-b3b1-4daa-b933-92bbd297d82a None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:37.185884 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a95a8d0a-b3b1-4daa-b933-92bbd297d82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:37.212609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3263df3f-3c2b-49a6-9dbe-70ce720f4c48 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:37.258554 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5f5de-b39e-4988-bc82-cea4e5ac61ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:37.258767 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5f5de-b39e-4988-bc82-cea4e5ac61ec None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:37.259015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5f5de-b39e-4988-bc82-cea4e5ac61ec None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:37.259496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5f5de-b39e-4988-bc82-cea4e5ac61ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:37.260178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5f5de-b39e-4988-bc82-cea4e5ac61ec None admin] RBAC: Adding query filter params (role.id=64c60e8079b94f7b9cf2473b4b75f5a5, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=e0c76478b6694782b896cdf9cda32d9c) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:37.260372 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5f5de-b39e-4988-bc82-cea4e5ac61ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:37.263106 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5f5de-b39e-4988-bc82-cea4e5ac61ec None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:37.269206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 453/913] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:00:37 2022] GET /identity/v3/role_assignments?user.id=e0c76478b6694782b896cdf9cda32d9c&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=64c60e8079b94f7b9cf2473b4b75f5a5 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:37.274559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 461/914] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:00:37.550599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5f5de-b39e-4988-bc82-cea4e5ac61ec None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:37.652184 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:37.660056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5f5de-b39e-4988-bc82-cea4e5ac61ec None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:37.695771 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94734d-2dee-4317-8d8b-ddae43818f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:37.696404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94734d-2dee-4317-8d8b-ddae43818f28 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:37.696685 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94734d-2dee-4317-8d8b-ddae43818f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:37.696956 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94734d-2dee-4317-8d8b-ddae43818f28 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:37.697534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94734d-2dee-4317-8d8b-ddae43818f28 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:37.698931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94734d-2dee-4317-8d8b-ddae43818f28 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:37.708502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-8c94734d-2dee-4317-8d8b-ddae43818f28 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:00:37.708502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:37.708502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:00:37.708502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:00:37.708502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:00:37.708502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:00:37.708502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:00:37.708502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:37.708502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:00:37.708502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:37.708502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:37.708502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:37.708502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:37.708502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:37.708502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:37.708502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:37.708502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:37.709524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:37.709524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:37.709524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:37.709524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:37.709524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:37.709524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:37.709524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:00:37.709524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:00:37.709524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:00:37.709524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:00:37.709524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:37.709524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:37.709524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:37.709524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:37.709524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:37.709524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:37.710392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:37.710392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:37.710392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:37.710392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:37.710392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:37.710392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:37.710392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:37.710392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:37.710392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:37.710392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:37.710392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:00:37.710392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:37.710392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:00:37.710392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:37.714045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 454/915] 10.222.0.73 () {56 vars in 1114 bytes} [Mon Aug 29 20:00:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:37.716017 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a8d0a-b3b1-4daa-b933-92bbd297d82a None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:37.819628 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:37.825820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a8d0a-b3b1-4daa-b933-92bbd297d82a None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:37.863545 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba589af-e460-421c-80e7-4a669561be39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:37.864168 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba589af-e460-421c-80e7-4a669561be39 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:37.864447 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba589af-e460-421c-80e7-4a669561be39 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:37.864723 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba589af-e460-421c-80e7-4a669561be39 None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:37.865361 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba589af-e460-421c-80e7-4a669561be39 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:37.865671 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba589af-e460-421c-80e7-4a669561be39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:37.867076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba589af-e460-421c-80e7-4a669561be39 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:37.877357 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 462/916] 10.222.0.73 () {56 vars in 1114 bytes} [Mon Aug 29 20:00:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:37.878720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94734d-2dee-4317-8d8b-ddae43818f28 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:37.986112 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94734d-2dee-4317-8d8b-ddae43818f28 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:38.022602 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f27cac-b419-48b1-98d1-5b0aa0bb9b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:38.023217 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f27cac-b419-48b1-98d1-5b0aa0bb9b81 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:38.023499 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f27cac-b419-48b1-98d1-5b0aa0bb9b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:38.023774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f27cac-b419-48b1-98d1-5b0aa0bb9b81 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:38.024412 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f27cac-b419-48b1-98d1-5b0aa0bb9b81 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:38.024717 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f27cac-b419-48b1-98d1-5b0aa0bb9b81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:38.026112 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f27cac-b419-48b1-98d1-5b0aa0bb9b81 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:38.036949 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 455/917] 10.222.0.73 () {56 vars in 1114 bytes} [Mon Aug 29 20:00:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:38.038318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba589af-e460-421c-80e7-4a669561be39 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:38.147048 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba589af-e460-421c-80e7-4a669561be39 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:38.183377 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c230a-ee35-4414-b10c-61e5fc705d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:38.183997 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c230a-ee35-4414-b10c-61e5fc705d8e None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:38.184277 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c230a-ee35-4414-b10c-61e5fc705d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:38.184740 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c230a-ee35-4414-b10c-61e5fc705d8e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:38.186273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7aba9-6cb1-4667-9bd2-e8862b239c88 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:38.186553 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7aba9-6cb1-4667-9bd2-e8862b239c88 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:38.186977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c230a-ee35-4414-b10c-61e5fc705d8e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:38.187085 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7aba9-6cb1-4667-9bd2-e8862b239c88 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:38.188606 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 456/918] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:00:38.188837 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c230a-ee35-4414-b10c-61e5fc705d8e None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:38.196054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c31ad-1a86-46b7-ba08-bbd667fa7860 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:38.196624 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c31ad-1a86-46b7-ba08-bbd667fa7860 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:38.197106 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c31ad-1a86-46b7-ba08-bbd667fa7860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:38.213626 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 463/919] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:00:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:00:38.229130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-0a5c31ad-1a86-46b7-ba08-bbd667fa7860 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:38.230593 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-0a5c31ad-1a86-46b7-ba08-bbd667fa7860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:38.253339 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:38.320828 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 457/920] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:38.595144 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c230a-ee35-4414-b10c-61e5fc705d8e None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:38.614984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:38.705039 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c230a-ee35-4414-b10c-61e5fc705d8e None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:38.741847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f4ed2-5ed5-486f-8716-4d2aa324e559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:38.742481 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f4ed2-5ed5-486f-8716-4d2aa324e559 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:38.742764 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f4ed2-5ed5-486f-8716-4d2aa324e559 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:38.743048 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f4ed2-5ed5-486f-8716-4d2aa324e559 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:38.743626 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f4ed2-5ed5-486f-8716-4d2aa324e559 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:38.745051 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f4ed2-5ed5-486f-8716-4d2aa324e559 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:38.755816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-b05f4ed2-5ed5-486f-8716-4d2aa324e559 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:00:38.755816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:38.755816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:00:38.755816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:00:38.755816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:00:38.755816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:00:38.755816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:00:38.755816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:38.755816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:00:38.755816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:38.755816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:38.755816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:38.755816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:38.755816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:38.755816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:38.755816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:38.755816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:38.756927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:38.756927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:38.756927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:38.756927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:38.756927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:38.756927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:38.756927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:00:38.756927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:00:38.756927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:00:38.756927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:00:38.756927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:38.756927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:38.756927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:38.756927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:38.756927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:38.756927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:38.757929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:38.757929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:38.757929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:38.757929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:38.757929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:38.757929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:38.757929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:38.757929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:38.757929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:38.757929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:38.757929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:00:38.757929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:38.757929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:00:38.757929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:38.761827 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 464/921] 10.222.0.73 () {56 vars in 1126 bytes} [Mon Aug 29 20:00:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:38.763345 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c31ad-1a86-46b7-ba08-bbd667fa7860 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:38.871593 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c31ad-1a86-46b7-ba08-bbd667fa7860 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:38.909556 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258dd9d-a2d5-45f3-a1c5-b718b0b4bc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:38.910218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258dd9d-a2d5-45f3-a1c5-b718b0b4bc55 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:38.910500 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258dd9d-a2d5-45f3-a1c5-b718b0b4bc55 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:38.910775 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258dd9d-a2d5-45f3-a1c5-b718b0b4bc55 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:38.911417 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6258dd9d-a2d5-45f3-a1c5-b718b0b4bc55 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:38.911723 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6258dd9d-a2d5-45f3-a1c5-b718b0b4bc55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:38.913100 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6258dd9d-a2d5-45f3-a1c5-b718b0b4bc55 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:38.923291 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 458/922] 10.222.0.73 () {56 vars in 1122 bytes} [Mon Aug 29 20:00:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:38.924856 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f4ed2-5ed5-486f-8716-4d2aa324e559 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:39.022452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:39.033332 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f4ed2-5ed5-486f-8716-4d2aa324e559 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:39.068297 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe671302-acab-498d-9fe1-5a7635a7b444 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:39.068629 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe671302-acab-498d-9fe1-5a7635a7b444 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:39.068967 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe671302-acab-498d-9fe1-5a7635a7b444 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:39.070312 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0213f-2c44-4a75-ba2d-fe9543b50991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:39.070694 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 459/923] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:00:39.070996 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0213f-2c44-4a75-ba2d-fe9543b50991 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:39.071278 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0213f-2c44-4a75-ba2d-fe9543b50991 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:39.071555 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0213f-2c44-4a75-ba2d-fe9543b50991 None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:39.072251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0213f-2c44-4a75-ba2d-fe9543b50991 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:39.072558 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0213f-2c44-4a75-ba2d-fe9543b50991 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:39.073956 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0213f-2c44-4a75-ba2d-fe9543b50991 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:39.075641 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7049ec3-bf9c-48a4-b7b6-3ddafe36496f None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:39.075930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7049ec3-bf9c-48a4-b7b6-3ddafe36496f None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:39.076208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7049ec3-bf9c-48a4-b7b6-3ddafe36496f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:39.085032 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 465/924] 10.222.0.73 () {56 vars in 1122 bytes} [Mon Aug 29 20:00:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:39.087195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0213f-2c44-4a75-ba2d-fe9543b50991 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:39.105045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-b7049ec3-bf9c-48a4-b7b6-3ddafe36496f None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:39.106299 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-b7049ec3-bf9c-48a4-b7b6-3ddafe36496f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:39.125181 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:39.190193 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 460/925] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:39.217195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0213f-2c44-4a75-ba2d-fe9543b50991 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:39.246433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf792da4-b94f-49f0-b50d-df4d069c44fa None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:39.246758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf792da4-b94f-49f0-b50d-df4d069c44fa None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:39.246992 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf792da4-b94f-49f0-b50d-df4d069c44fa None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:39.249175 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 461/926] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:00:39.254740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89474f8-f828-44b4-974a-081eefd018f9 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:39.255027 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89474f8-f828-44b4-974a-081eefd018f9 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:39.255343 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89474f8-f828-44b4-974a-081eefd018f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:39.255890 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0e21a-db90-48a6-ac93-d0da119f48ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:39.256678 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0e21a-db90-48a6-ac93-d0da119f48ef None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:39.257114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0e21a-db90-48a6-ac93-d0da119f48ef None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:39.257539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0e21a-db90-48a6-ac93-d0da119f48ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:39.258418 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0e21a-db90-48a6-ac93-d0da119f48ef None admin] RBAC: Adding query filter params (interface=public, service_id=306bd2e233cc4dc185826aee41f43c3f, region_id=RegionOne) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:39.258872 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0e21a-db90-48a6-ac93-d0da119f48ef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:39.260463 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0e21a-db90-48a6-ac93-d0da119f48ef None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:39.268234 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 466/927] 10.222.0.73 () {56 vars in 1251 bytes} [Mon Aug 29 20:00:39 2022] GET /identity/v3/endpoints?service_id=306bd2e233cc4dc185826aee41f43c3f&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:39.273117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0e21a-db90-48a6-ac93-d0da119f48ef None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:39.278263 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-d89474f8-f828-44b4-974a-081eefd018f9 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:39.280261 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d89474f8-f828-44b4-974a-081eefd018f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:39.286401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:39.356705 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 462/928] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:39.396935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0e21a-db90-48a6-ac93-d0da119f48ef None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:39.432350 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da79143-bc39-4de9-b1a2-94efd286bd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:39.432981 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da79143-bc39-4de9-b1a2-94efd286bd8f None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:39.433305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da79143-bc39-4de9-b1a2-94efd286bd8f None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:39.433577 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da79143-bc39-4de9-b1a2-94efd286bd8f None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:39.434264 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da79143-bc39-4de9-b1a2-94efd286bd8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:39.435686 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da79143-bc39-4de9-b1a2-94efd286bd8f None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:39.443484 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 467/929] 10.222.0.73 () {56 vars in 1087 bytes} [Mon Aug 29 20:00:39 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:00:39.477889 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89474f8-f828-44b4-974a-081eefd018f9 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:39.559711 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:39.594601 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89474f8-f828-44b4-974a-081eefd018f9 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:39.634108 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02476d7a-21cf-49f0-b58e-7105cf67a9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:39.634749 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02476d7a-21cf-49f0-b58e-7105cf67a9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:39.635029 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02476d7a-21cf-49f0-b58e-7105cf67a9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:39.635301 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02476d7a-21cf-49f0-b58e-7105cf67a9b5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:39.635892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02476d7a-21cf-49f0-b58e-7105cf67a9b5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:39.637259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02476d7a-21cf-49f0-b58e-7105cf67a9b5 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:39.646387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-02476d7a-21cf-49f0-b58e-7105cf67a9b5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:00:39.646387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:39.646387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:00:39.646387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:00:39.646387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:00:39.646387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:00:39.646387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:00:39.646387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:39.646387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:00:39.646387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:39.646387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:39.646387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:39.646387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:39.646387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:39.646387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:39.646387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:39.647404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:39.647404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:39.647404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:39.647404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:39.647404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:39.647404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:39.647404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:39.647404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:00:39.647404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:00:39.647404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:00:39.647404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:00:39.647404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:39.647404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:39.647404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:39.647404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:39.647404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:39.647404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:39.648301 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:39.648301 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:39.648301 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:39.648301 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:39.648301 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:39.648301 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:39.648301 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:39.648301 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:39.648301 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:39.648301 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:39.648301 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:00:39.648301 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:39.648301 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:00:39.648301 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:39.649035 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 463/930] 10.222.0.73 () {56 vars in 1132 bytes} [Mon Aug 29 20:00:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:39.653951 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da79143-bc39-4de9-b1a2-94efd286bd8f None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:39.663516 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02476d7a-21cf-49f0-b58e-7105cf67a9b5 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:39.759308 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:39.788092 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02476d7a-21cf-49f0-b58e-7105cf67a9b5 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:39.795106 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da79143-bc39-4de9-b1a2-94efd286bd8f None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:39.833689 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb032e4-982e-43c9-b383-342495c2f957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:39.834753 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb032e4-982e-43c9-b383-342495c2f957 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:39.834889 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa257f3-0f49-4dbb-b51f-17ac82b442f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:39.835375 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb032e4-982e-43c9-b383-342495c2f957 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:39.835490 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa257f3-0f49-4dbb-b51f-17ac82b442f6 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:39.835711 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa257f3-0f49-4dbb-b51f-17ac82b442f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:39.835985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa257f3-0f49-4dbb-b51f-17ac82b442f6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:39.836394 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb032e4-982e-43c9-b383-342495c2f957 None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:39.837358 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb032e4-982e-43c9-b383-342495c2f957 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:39.837909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb032e4-982e-43c9-b383-342495c2f957 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:39.839057 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:39.839881 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb032e4-982e-43c9-b383-342495c2f957 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:39.847526 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 468/931] 10.222.0.73 () {56 vars in 1126 bytes} [Mon Aug 29 20:00:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:39.851663 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa257f3-0f49-4dbb-b51f-17ac82b442f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:00:39.852883 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb032e4-982e-43c9-b383-342495c2f957 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:39.857846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-3aa257f3-0f49-4dbb-b51f-17ac82b442f6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:00:39.857846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:39.857846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:39.857846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:39.857846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:39.857846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:39.857846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:39.857846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:39.857846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:39.857846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:39.857846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:39.857846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:39.857846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:00:39.857846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:00:39.857846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:00:39.857846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:00:39.859338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:00:39.859338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:00:39.859338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:00:39.859338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:00:39.859338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:39.859338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:39.859338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:39.859338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:39.859338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:39.859338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:39.859338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:39.859338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:39.859338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:39.859338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:39.859338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:39.859338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:39.860188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:39.860188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:39.860188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:39.860188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:39.860188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:00:39.860188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:00:39.860188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:00:39.860188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:00:39.860188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:00:39.860188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:00:39.860188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:00:39.860188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:39.860188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 464/932] 10.222.0.73 () {56 vars in 1120 bytes} [Mon Aug 29 20:00:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:00:39.865625 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa257f3-0f49-4dbb-b51f-17ac82b442f6 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:39.979641 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb032e4-982e-43c9-b383-342495c2f957 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:39.995708 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa257f3-0f49-4dbb-b51f-17ac82b442f6 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:40.022379 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f644532-d9a1-4931-8f49-cc5695131f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:40.022997 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f644532-d9a1-4931-8f49-cc5695131f17 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:40.023272 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f644532-d9a1-4931-8f49-cc5695131f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:40.023542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f644532-d9a1-4931-8f49-cc5695131f17 None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:40.024173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f644532-d9a1-4931-8f49-cc5695131f17 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:40.024479 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f644532-d9a1-4931-8f49-cc5695131f17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:40.025891 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f644532-d9a1-4931-8f49-cc5695131f17 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:40.028150 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:40.037078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 469/933] 10.222.0.73 () {56 vars in 1126 bytes} [Mon Aug 29 20:00:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:40.037921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f644532-d9a1-4931-8f49-cc5695131f17 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:40.049680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f916e43-51af-4641-bf95-f5b861e48b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:40.050369 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f916e43-51af-4641-bf95-f5b861e48b90 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:40.050649 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f916e43-51af-4641-bf95-f5b861e48b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:40.050921 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f916e43-51af-4641-bf95-f5b861e48b90 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:40.051651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f916e43-51af-4641-bf95-f5b861e48b90 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:40.051955 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f916e43-51af-4641-bf95-f5b861e48b90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:40.054294 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f916e43-51af-4641-bf95-f5b861e48b90 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:40.066333 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 465/934] 10.222.0.73 () {56 vars in 1115 bytes} [Mon Aug 29 20:00:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:40.071751 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f916e43-51af-4641-bf95-f5b861e48b90 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:40.156680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:40.163997 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f644532-d9a1-4931-8f49-cc5695131f17 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:40.200286 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e824cc3-57fa-4549-b87c-487a1c009f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:40.200920 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e824cc3-57fa-4549-b87c-487a1c009f6d None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:40.201192 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e824cc3-57fa-4549-b87c-487a1c009f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:40.202087 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e824cc3-57fa-4549-b87c-487a1c009f6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:40.202985 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e824cc3-57fa-4549-b87c-487a1c009f6d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:40.204568 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e824cc3-57fa-4549-b87c-487a1c009f6d None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:40.207756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f916e43-51af-4641-bf95-f5b861e48b90 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:40.234382 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 470/935] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:00:40 2022] POST /identity/v3/endpoints => generated 347 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:00:40.256230 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2e0bc-88bf-4964-975e-d91aedbbddd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:40.257298 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2e0bc-88bf-4964-975e-d91aedbbddd4 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:40.257901 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2e0bc-88bf-4964-975e-d91aedbbddd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:40.258383 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2e0bc-88bf-4964-975e-d91aedbbddd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:40.272349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a2e0bc-88bf-4964-975e-d91aedbbddd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:40.274324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a2e0bc-88bf-4964-975e-d91aedbbddd4 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:40.292677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-14a2e0bc-88bf-4964-975e-d91aedbbddd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:40.292677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:40.292677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:40.292677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:40.292677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:40.292677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:40.292677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:40.292677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:40.292677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:40.292677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:40.292677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:40.292677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:40.292677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:40.292677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:40.292677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:40.292677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:40.293745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:40.293745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:40.293745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:40.293745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:40.293745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:40.293745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:40.293745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:40.293745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:40.293745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:40.293745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:40.293745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:40.293745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:40.293745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:40.293745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:40.293745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:40.293745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:40.293745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:40.294661 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:40.294661 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:40.294661 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:40.294831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f7335-dedd-4f2c-9ae1-7fd72aa61a9f None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:40.294911 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 466/936] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:40.294984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f7335-dedd-4f2c-9ae1-7fd72aa61a9f None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:40.294984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f7335-dedd-4f2c-9ae1-7fd72aa61a9f None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:40.296507 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 471/937] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:40.301630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2e0bc-88bf-4964-975e-d91aedbbddd4 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:40.302389 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6991d-b78c-4b49-84a6-5fb7df068323 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:40.302689 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6991d-b78c-4b49-84a6-5fb7df068323 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:40.302970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6991d-b78c-4b49-84a6-5fb7df068323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:40.328213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-5cf6991d-b78c-4b49-84a6-5fb7df068323 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:40.329843 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-5cf6991d-b78c-4b49-84a6-5fb7df068323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:40.393401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:40.412997 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 472/938] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:00:40.435047 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2e0bc-88bf-4964-975e-d91aedbbddd4 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:40.444508 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:40.477902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed1c9f-e265-4bb0-9b4c-2c36dd71fb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:40.478521 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed1c9f-e265-4bb0-9b4c-2c36dd71fb02 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:40.478802 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed1c9f-e265-4bb0-9b4c-2c36dd71fb02 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:40.479076 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed1c9f-e265-4bb0-9b4c-2c36dd71fb02 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:40.479733 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed1c9f-e265-4bb0-9b4c-2c36dd71fb02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:40.480031 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed1c9f-e265-4bb0-9b4c-2c36dd71fb02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:40.481402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed1c9f-e265-4bb0-9b4c-2c36dd71fb02 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:40.495812 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 467/939] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:40.501856 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf6991d-b78c-4b49-84a6-5fb7df068323 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:40.612195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf6991d-b78c-4b49-84a6-5fb7df068323 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:40.648947 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e25798d-1b7d-4b81-90b9-928320b6267e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:40.649615 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25798d-1b7d-4b81-90b9-928320b6267e None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:40.649928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25798d-1b7d-4b81-90b9-928320b6267e None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:40.650201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25798d-1b7d-4b81-90b9-928320b6267e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:40.659284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e25798d-1b7d-4b81-90b9-928320b6267e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:40.660744 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e25798d-1b7d-4b81-90b9-928320b6267e None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:40.673849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-5e25798d-1b7d-4b81-90b9-928320b6267e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:00:40.673849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:40.673849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:40.673849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:40.673849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:40.673849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:40.673849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:40.673849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:40.673849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:40.673849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:40.673849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:40.673849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:40.673849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:00:40.673849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:00:40.673849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:00:40.673849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:00:40.675069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:40.675069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:40.675069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:40.675069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:40.675069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:40.675069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:40.675069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:40.675069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:40.675069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:40.675069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:40.675069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:40.675069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:40.675069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:40.675069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:40.675069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:40.675069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:40.675069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:40.675069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:40.676245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:40.676245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:40.676245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:00:40.676245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:00:40.676245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:00:40.676245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:00:40.676245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:00:40.676245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:00:40.676245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:00:40.676245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:40.679027 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 473/940] 10.222.0.73 () {56 vars in 1117 bytes} [Mon Aug 29 20:00:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:40.681922 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed1c9f-e265-4bb0-9b4c-2c36dd71fb02 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:40.684562 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e25798d-1b7d-4b81-90b9-928320b6267e None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:40.745651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:40.810703 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e25798d-1b7d-4b81-90b9-928320b6267e None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:40.822521 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed1c9f-e265-4bb0-9b4c-2c36dd71fb02 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:40.851465 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c7c46-5065-4c77-810d-b283565b6e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:40.852414 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c7c46-5065-4c77-810d-b283565b6e53 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:40.852895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c7c46-5065-4c77-810d-b283565b6e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:40.853360 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c7c46-5065-4c77-810d-b283565b6e53 None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:40.854324 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c7c46-5065-4c77-810d-b283565b6e53 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:40.854841 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c7c46-5065-4c77-810d-b283565b6e53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:40.856479 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c7c46-5065-4c77-810d-b283565b6e53 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:40.869626 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150aa7ad-721f-4735-825f-74f0b2213cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:40.870271 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150aa7ad-721f-4735-825f-74f0b2213cc9 None admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:40.870553 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150aa7ad-721f-4735-825f-74f0b2213cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:40.870827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150aa7ad-721f-4735-825f-74f0b2213cc9 None admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:40.871480 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150aa7ad-721f-4735-825f-74f0b2213cc9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:40.872909 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150aa7ad-721f-4735-825f-74f0b2213cc9 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:40.876712 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: WARNING keystone.server.flask.common [None req-150aa7ad-721f-4735-825f-74f0b2213cc9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 29 20:00:40.878903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 474/941] 10.222.0.73 () {56 vars in 1149 bytes} [Mon Aug 29 20:00:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:40.885314 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c7c46-5065-4c77-810d-b283565b6e53 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:40.902083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.sql.core [None req-150aa7ad-721f-4735-825f-74f0b2213cc9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 20:00:40.902083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 29 20:00:40.902083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [parameters: {'id': 'f887aedd07fe418cb0b66605dd49e84b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 29 20:00:40.902083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81781) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 20:00:40.925616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-150aa7ad-721f-4735-825f-74f0b2213cc9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f887aedd07fe418cb0b66605dd49e84b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f887aedd07fe418cb0b66605dd49e84b. Aug 29 20:00:40.925616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:40.925616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 20:00:40.925616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 20:00:40.925616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 20:00:40.925616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 20:00:40.925616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 20:00:40.925616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 20:00:40.925616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 20:00:40.925616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application conn.query(q) Aug 29 20:00:40.925616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 20:00:40.925616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 20:00:40.925616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 20:00:40.926528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application result.read() Aug 29 20:00:40.926528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 20:00:40.926528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 20:00:40.926528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 20:00:40.926528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 20:00:40.926528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 20:00:40.926528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 20:00:40.926528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 20:00:40.926528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 20:00:40.926528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 20:00:40.926528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:40.926528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 20:00:40.926528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:40.926528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:40.926528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 20:00:40.926528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 20:00:40.926528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 29 20:00:40.926528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 29 20:00:40.927478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 20:00:40.927478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application next(self.gen) Aug 29 20:00:40.927478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 20:00:40.927478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application yield resource Aug 29 20:00:40.927478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 20:00:40.927478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application next(self.gen) Aug 29 20:00:40.927478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 20:00:40.927478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 20:00:40.927478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 20:00:40.927478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 20:00:40.927478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 20:00:40.927478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise self.value Aug 29 20:00:40.927478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 20:00:40.927478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 20:00:40.927478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 20:00:40.927478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application session.commit() Aug 29 20:00:40.927478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 20:00:40.928381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 20:00:40.928381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 20:00:40.928381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 20:00:40.928381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 20:00:40.928381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self.session.flush() Aug 29 20:00:40.928381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 20:00:40.928381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 20:00:40.928381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 20:00:40.928381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 20:00:40.928381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 20:00:40.928381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application compat.raise_( Aug 29 20:00:40.928381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 20:00:40.928381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception Aug 29 20:00:40.928381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 20:00:40.928381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 20:00:40.928381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 20:00:40.928381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 20:00:40.929349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 20:00:40.929349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 20:00:40.929349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 20:00:40.929349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 20:00:40.929349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 29 20:00:40.929349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 29 20:00:40.929349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 20:00:40.929349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 20:00:40.929349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 20:00:40.929349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 20:00:40.929349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 20:00:40.929349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 20:00:40.929349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 20:00:40.929349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 20:00:40.929349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 20:00:40.931623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 20:00:40.931623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 20:00:40.931623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception Aug 29 20:00:40.931623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 20:00:40.931623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 20:00:40.931623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 20:00:40.931623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 20:00:40.931623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 20:00:40.931623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 20:00:40.931623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 20:00:40.931623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application conn.query(q) Aug 29 20:00:40.931623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 20:00:40.931623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 20:00:40.931623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 20:00:40.931623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application result.read() Aug 29 20:00:40.931623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 20:00:40.931623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 20:00:40.933761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 20:00:40.933761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 20:00:40.933761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 20:00:40.933761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 20:00:40.933761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 20:00:40.933761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 20:00:40.933761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 20:00:40.933761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 29 20:00:40.933761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [parameters: {'id': 'f887aedd07fe418cb0b66605dd49e84b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 29 20:00:40.933761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 20:00:40.933761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:40.933761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:40.933761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:40.933761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:40.933761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 29 20:00:40.935684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 29 20:00:40.935684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 20:00:40.935684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 20:00:40.935684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 29 20:00:40.935684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:40.935684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:40.935684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:40.935684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:40.935684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:40.935684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:40.935684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:40.935684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:40.935684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:40.935684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:40.935684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:40.935684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:40.935684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:40.937892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:40.937892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 29 20:00:40.937892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 29 20:00:40.937892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:40.937892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:40.937892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 29 20:00:40.937892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 29 20:00:40.937892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f887aedd07fe418cb0b66605dd49e84b. Aug 29 20:00:40.937892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:40.937892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 468/942] 10.222.0.73 () {60 vars in 1140 bytes} [Mon Aug 29 20:00:40 2022] POST /identity/v3/projects => generated 262 bytes in 246 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:00:40.937892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150aa7ad-721f-4735-825f-74f0b2213cc9 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:40.973917 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:41.019160 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c7c46-5065-4c77-810d-b283565b6e53 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:41.022040 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:41.058955 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18384dd-36b3-4831-ad94-ce11294fea7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:41.059976 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18384dd-36b3-4831-ad94-ce11294fea7e None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:41.060491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18384dd-36b3-4831-ad94-ce11294fea7e None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:41.061088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18384dd-36b3-4831-ad94-ce11294fea7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:41.062308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150aa7ad-721f-4735-825f-74f0b2213cc9 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:41.072620 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18384dd-36b3-4831-ad94-ce11294fea7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:41.074415 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18384dd-36b3-4831-ad94-ce11294fea7e None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:41.095141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-b18384dd-36b3-4831-ad94-ce11294fea7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:41.095141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:41.095141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:41.095141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:41.095141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:41.095141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:41.095141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:41.095141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:41.095141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:41.095141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:41.095141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:41.095141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:41.095141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:41.095141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:41.095141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:41.095141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:41.096078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:41.096078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:41.096078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:41.096078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:41.096078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:41.096078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:41.096078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:41.096078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:41.096078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:41.096078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:41.096078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:41.096078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:41.096078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:41.096078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:41.096078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:41.096078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:41.096078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:41.096956 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:41.096956 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:41.096956 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:41.100862 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099190e5-7568-41d2-a20d-cd0c2149c1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:41.100862 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099190e5-7568-41d2-a20d-cd0c2149c1cc None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:41.101071 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099190e5-7568-41d2-a20d-cd0c2149c1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:41.101331 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099190e5-7568-41d2-a20d-cd0c2149c1cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:41.102478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 475/943] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:41.109286 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18384dd-36b3-4831-ad94-ce11294fea7e None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:41.113741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099190e5-7568-41d2-a20d-cd0c2149c1cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:41.115340 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099190e5-7568-41d2-a20d-cd0c2149c1cc None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:41.129426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-099190e5-7568-41d2-a20d-cd0c2149c1cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:41.129426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:41.129426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:41.129426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:41.129426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:41.129426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:41.129426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:41.129426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:41.129426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:41.129426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:41.129426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:41.129426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:41.129426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:41.129426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:41.129426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:41.130426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:41.130426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:41.130426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:41.130426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:41.130426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:41.130426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:41.130426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:41.130426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:41.130426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:41.130426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:41.130426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:41.130426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:41.130426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:41.130426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:41.130426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:41.130426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:41.130426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:41.131350 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:41.131350 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:41.131350 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:41.131350 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:41.131350 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:41.131350 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:41.131350 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:41.131350 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:41.131350 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 469/944] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:41.136312 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099190e5-7568-41d2-a20d-cd0c2149c1cc None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:41.239123 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18384dd-36b3-4831-ad94-ce11294fea7e None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:41.243472 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:41.270281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099190e5-7568-41d2-a20d-cd0c2149c1cc None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:41.284375 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762aa07f-20cd-4243-a2f3-ceee16db8c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:41.285413 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762aa07f-20cd-4243-a2f3-ceee16db8c1e None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:41.285936 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762aa07f-20cd-4243-a2f3-ceee16db8c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:41.286689 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762aa07f-20cd-4243-a2f3-ceee16db8c1e None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:41.287654 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762aa07f-20cd-4243-a2f3-ceee16db8c1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:41.288189 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762aa07f-20cd-4243-a2f3-ceee16db8c1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:41.289878 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762aa07f-20cd-4243-a2f3-ceee16db8c1e None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:41.305595 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 476/945] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:41.310804 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762aa07f-20cd-4243-a2f3-ceee16db8c1e None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:41.315823 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac673c0-b743-42cb-9194-5959e4d5ab12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:41.316462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac673c0-b743-42cb-9194-5959e4d5ab12 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:41.316813 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac673c0-b743-42cb-9194-5959e4d5ab12 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:41.317160 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac673c0-b743-42cb-9194-5959e4d5ab12 None admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:41.317957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac673c0-b743-42cb-9194-5959e4d5ab12 None admin] RBAC: Adding query filter params (name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:41.318330 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac673c0-b743-42cb-9194-5959e4d5ab12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:41.319834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac673c0-b743-42cb-9194-5959e4d5ab12 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:41.333882 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:41.344090 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 470/946] 10.222.0.73 () {56 vars in 1112 bytes} [Mon Aug 29 20:00:41 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:41.433848 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762aa07f-20cd-4243-a2f3-ceee16db8c1e None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:41.477946 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de67631-d887-4a1e-81f7-5b27244356f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:41.479143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de67631-d887-4a1e-81f7-5b27244356f9 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:41.479734 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de67631-d887-4a1e-81f7-5b27244356f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:41.480194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de67631-d887-4a1e-81f7-5b27244356f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:41.488884 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de67631-d887-4a1e-81f7-5b27244356f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:41.490663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de67631-d887-4a1e-81f7-5b27244356f9 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:41.503874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-5de67631-d887-4a1e-81f7-5b27244356f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:41.503874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:41.503874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:41.503874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:41.503874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:41.503874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:41.503874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:41.503874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:41.503874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:41.503874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:41.503874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:41.503874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:41.503874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:41.503874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:41.503874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:41.504801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:41.504801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:41.504801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:41.504801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:41.504801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:41.504801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:41.504801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:41.504801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:41.504801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:41.504801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:41.504801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:41.504801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:41.504801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:41.504801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:41.504801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:41.504801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:41.504801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:41.505686 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:41.505686 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:41.505686 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:41.505686 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:41.505686 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:41.505686 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:41.505686 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:41.505686 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:41.507182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 477/947] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:41.512902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac673c0-b743-42cb-9194-5959e4d5ab12 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:41.547763 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90982b72-9bc6-4411-9b02-90fa3b5a862e None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:41.548094 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90982b72-9bc6-4411-9b02-90fa3b5a862e None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:41.548321 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90982b72-9bc6-4411-9b02-90fa3b5a862e None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:41.549824 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 478/948] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:00:41.555466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cad0f6-5767-4be6-b818-1906932af147 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:41.556016 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cad0f6-5767-4be6-b818-1906932af147 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:41.556490 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cad0f6-5767-4be6-b818-1906932af147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:41.576534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-d9cad0f6-5767-4be6-b818-1906932af147 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:41.578752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-d9cad0f6-5767-4be6-b818-1906932af147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:41.634640 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac673c0-b743-42cb-9194-5959e4d5ab12 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:41.673368 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 479/949] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:00:41.675819 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f655e-b7cf-4a09-bff4-804b8d36cd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:41.676702 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f655e-b7cf-4a09-bff4-804b8d36cd6e None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:41.677188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f655e-b7cf-4a09-bff4-804b8d36cd6e None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:41.677659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f655e-b7cf-4a09-bff4-804b8d36cd6e None admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:41.678588 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f655e-b7cf-4a09-bff4-804b8d36cd6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:41.679139 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f655e-b7cf-4a09-bff4-804b8d36cd6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:41.680788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f655e-b7cf-4a09-bff4-804b8d36cd6e None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:41.694720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 471/950] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:00:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:41.701827 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cad0f6-5767-4be6-b818-1906932af147 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:41.709522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:41.810834 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cad0f6-5767-4be6-b818-1906932af147 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:41.847987 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a94bcb-c166-4302-b626-bf53918974ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:41.848648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a94bcb-c166-4302-b626-bf53918974ef None admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:41.848923 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a94bcb-c166-4302-b626-bf53918974ef None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:41.849193 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a94bcb-c166-4302-b626-bf53918974ef None admin] PATH_INFO: `/v3/projects/2f5ec615d98548159487c97cead336a0/users/e0c76478b6694782b896cdf9cda32d9c/roles/64c60e8079b94f7b9cf2473b4b75f5a5` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:41.851436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a94bcb-c166-4302-b626-bf53918974ef None admin] RBAC: Authorizing `identity:create_grant(project_id=2f5ec615d98548159487c97cead336a0, user_id=e0c76478b6694782b896cdf9cda32d9c, role_id=64c60e8079b94f7b9cf2473b4b75f5a5)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:41.852876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a94bcb-c166-4302-b626-bf53918974ef None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:41.868539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 480/951] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 20:00:41 2022] PUT /identity/v3/projects/2f5ec615d98548159487c97cead336a0/users/e0c76478b6694782b896cdf9cda32d9c/roles/64c60e8079b94f7b9cf2473b4b75f5a5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:00:41.954135 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f655e-b7cf-4a09-bff4-804b8d36cd6e None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:41.958224 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:42.082755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f655e-b7cf-4a09-bff4-804b8d36cd6e None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:42.122858 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0056f6b-35ab-47db-892d-234c800bd15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:42.123929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0056f6b-35ab-47db-892d-234c800bd15d None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:42.124378 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0056f6b-35ab-47db-892d-234c800bd15d None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:42.124854 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0056f6b-35ab-47db-892d-234c800bd15d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:42.125553 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0056f6b-35ab-47db-892d-234c800bd15d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:42.127231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0056f6b-35ab-47db-892d-234c800bd15d None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:42.136679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-c0056f6b-35ab-47db-892d-234c800bd15d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:00:42.136679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:42.136679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:00:42.136679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:00:42.136679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:00:42.136679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:00:42.136679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:00:42.136679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:42.136679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:00:42.136679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:42.136679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:42.136679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:42.136679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:42.136679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:42.136679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:42.136679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:42.136679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:42.137750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:42.137750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:42.137750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:42.137750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:42.137750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:42.137750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:42.137750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:00:42.137750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:00:42.137750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:00:42.137750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:00:42.137750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:42.137750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:42.137750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:42.137750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:42.137750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:42.137750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:42.138756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:42.138756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:42.138756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:42.138756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:42.138756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:42.138756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:42.138756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:42.138756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:42.138756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:42.138756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:42.138756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:00:42.138756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:42.138756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:00:42.138756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:42.138756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 472/952] 10.222.0.73 () {56 vars in 1126 bytes} [Mon Aug 29 20:00:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:42.147030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a94bcb-c166-4302-b626-bf53918974ef None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:42.216947 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:42.256599 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a94bcb-c166-4302-b626-bf53918974ef None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:42.296356 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd341f9c-bf46-4880-b12b-b9bd23597c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:42.296962 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd341f9c-bf46-4880-b12b-b9bd23597c29 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:42.297243 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd341f9c-bf46-4880-b12b-b9bd23597c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:42.297502 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd341f9c-bf46-4880-b12b-b9bd23597c29 None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:42.298165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd341f9c-bf46-4880-b12b-b9bd23597c29 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:42.298582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd341f9c-bf46-4880-b12b-b9bd23597c29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:42.300336 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd341f9c-bf46-4880-b12b-b9bd23597c29 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:42.305531 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 481/953] 10.222.0.73 () {56 vars in 1122 bytes} [Mon Aug 29 20:00:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:42.311535 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0056f6b-35ab-47db-892d-234c800bd15d None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:42.335621 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81aa5c-85b9-45bc-9ce4-af703c8f1bb5 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:42.336266 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81aa5c-85b9-45bc-9ce4-af703c8f1bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:42.336796 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81aa5c-85b9-45bc-9ce4-af703c8f1bb5 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:42.338732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 482/954] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:00:42.344611 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939e12c-3083-4bbf-a4e5-1597dd8d572a None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:42.345064 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939e12c-3083-4bbf-a4e5-1597dd8d572a None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:42.345502 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939e12c-3083-4bbf-a4e5-1597dd8d572a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:42.364685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-0939e12c-3083-4bbf-a4e5-1597dd8d572a None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:42.366249 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-0939e12c-3083-4bbf-a4e5-1597dd8d572a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:42.383544 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:42.422254 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0056f6b-35ab-47db-892d-234c800bd15d None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:42.460922 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 483/955] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:00:42.468242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd1a172-d9e4-4e2d-bfce-dea25eaba718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:42.468863 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1a172-d9e4-4e2d-bfce-dea25eaba718 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:42.469193 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1a172-d9e4-4e2d-bfce-dea25eaba718 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:42.469468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1a172-d9e4-4e2d-bfce-dea25eaba718 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:42.470145 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd1a172-d9e4-4e2d-bfce-dea25eaba718 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:42.470446 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd1a172-d9e4-4e2d-bfce-dea25eaba718 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:42.471918 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd1a172-d9e4-4e2d-bfce-dea25eaba718 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:42.482193 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 473/956] 10.222.0.73 () {56 vars in 1122 bytes} [Mon Aug 29 20:00:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:42.484103 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0939e12c-3083-4bbf-a4e5-1597dd8d572a None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:42.549894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:42.599393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0939e12c-3083-4bbf-a4e5-1597dd8d572a None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:42.638741 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95283e9d-4bb2-4d5e-bbf8-1339bc1dbbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:42.639788 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95283e9d-4bb2-4d5e-bbf8-1339bc1dbbb0 None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:42.640071 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95283e9d-4bb2-4d5e-bbf8-1339bc1dbbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:42.640350 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95283e9d-4bb2-4d5e-bbf8-1339bc1dbbb0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:42.641031 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95283e9d-4bb2-4d5e-bbf8-1339bc1dbbb0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:42.642437 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95283e9d-4bb2-4d5e-bbf8-1339bc1dbbb0 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:42.664475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 484/957] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:00:42 2022] POST /identity/v3/endpoints => generated 349 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:00:42.733413 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd1a172-d9e4-4e2d-bfce-dea25eaba718 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:42.849228 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd1a172-d9e4-4e2d-bfce-dea25eaba718 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:42.882637 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:42.897035 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de3e08-286a-49f8-b071-c2a693a7b136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:42.897620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de3e08-286a-49f8-b071-c2a693a7b136 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:42.898052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de3e08-286a-49f8-b071-c2a693a7b136 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:42.898265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de3e08-286a-49f8-b071-c2a693a7b136 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:42.898844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de3e08-286a-49f8-b071-c2a693a7b136 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:42.900190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de3e08-286a-49f8-b071-c2a693a7b136 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:42.910754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-01de3e08-286a-49f8-b071-c2a693a7b136 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:00:42.910754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:42.910754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:00:42.910754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:00:42.910754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:00:42.910754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:00:42.910754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:00:42.910754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:42.910754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:00:42.910754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:42.910754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:42.910754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:42.910754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:42.910754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:42.910754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:42.910754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:42.910754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:42.911808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:42.911808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:42.911808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:42.911808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:42.911808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:42.911808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:42.911808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:00:42.911808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:00:42.911808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:00:42.911808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:00:42.911808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:42.911808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:42.911808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:42.911808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:42.911808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:42.911808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:42.912980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:42.912980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:42.912980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:42.912980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:42.912980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:42.912980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:42.912980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:42.912980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:42.912980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:42.912980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:42.912980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:00:42.912980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:42.912980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:00:42.912980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:42.912980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 474/958] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:42.918857 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95283e9d-4bb2-4d5e-bbf8-1339bc1dbbb0 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:43.030739 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95283e9d-4bb2-4d5e-bbf8-1339bc1dbbb0 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:43.048520 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:43.074967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ff2e0-aee0-4569-b8b3-6175c3d75a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:43.075596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ff2e0-aee0-4569-b8b3-6175c3d75a01 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:43.075887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ff2e0-aee0-4569-b8b3-6175c3d75a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:43.076162 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ff2e0-aee0-4569-b8b3-6175c3d75a01 None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:43.076868 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ff2e0-aee0-4569-b8b3-6175c3d75a01 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:43.077738 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ff2e0-aee0-4569-b8b3-6175c3d75a01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:43.079140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ff2e0-aee0-4569-b8b3-6175c3d75a01 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:43.084721 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 485/959] 10.222.0.73 () {56 vars in 1112 bytes} [Mon Aug 29 20:00:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:43.092734 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de3e08-286a-49f8-b071-c2a693a7b136 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:43.210978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de3e08-286a-49f8-b071-c2a693a7b136 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:43.251878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad094305-bcc0-4db0-950b-e087e18b23a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:43.252764 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad094305-bcc0-4db0-950b-e087e18b23a8 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:43.253184 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad094305-bcc0-4db0-950b-e087e18b23a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:43.253594 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad094305-bcc0-4db0-950b-e087e18b23a8 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:43.254558 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad094305-bcc0-4db0-950b-e087e18b23a8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:43.255025 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad094305-bcc0-4db0-950b-e087e18b23a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:43.256527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad094305-bcc0-4db0-950b-e087e18b23a8 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:43.261780 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 475/960] 10.222.0.73 () {56 vars in 1112 bytes} [Mon Aug 29 20:00:43 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:43.442387 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b16f1-2839-4d7c-8cca-367a8a1c46d7 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:43.442672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b16f1-2839-4d7c-8cca-367a8a1c46d7 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:43.442949 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b16f1-2839-4d7c-8cca-367a8a1c46d7 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:43.444560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 486/961] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:43.449852 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e720d20-5a40-4250-948d-4d6e055b1e5f None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:43.450145 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e720d20-5a40-4250-948d-4d6e055b1e5f None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:43.450440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e720d20-5a40-4250-948d-4d6e055b1e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:43.469572 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-4e720d20-5a40-4250-948d-4d6e055b1e5f None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:43.471063 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-4e720d20-5a40-4250-948d-4d6e055b1e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:43.523654 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:43.542612 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 476/962] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:43.819325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b16f1-2839-4d7c-8cca-367a8a1c46d7 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:43.933874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b16f1-2839-4d7c-8cca-367a8a1c46d7 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:43.973335 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f5ca4-f751-4d8f-9ed4-885c7ceafad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:43.973986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f5ca4-f751-4d8f-9ed4-885c7ceafad7 None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:43.974271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f5ca4-f751-4d8f-9ed4-885c7ceafad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:43.974547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f5ca4-f751-4d8f-9ed4-885c7ceafad7 None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:43.975212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f5ca4-f751-4d8f-9ed4-885c7ceafad7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:43.976610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f5ca4-f751-4d8f-9ed4-885c7ceafad7 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:43.979255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: WARNING keystone.server.flask.common [None req-c38f5ca4-f751-4d8f-9ed4-885c7ceafad7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 29 20:00:43.992432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.sql.core [None req-c38f5ca4-f751-4d8f-9ed4-885c7ceafad7 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 20:00:43.992432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 29 20:00:43.992432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [parameters: {'user_id': 'ee935ad3da4d46c9a89a942500d5a7e5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 29 20:00:43.992432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81782) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 20:00:44.000137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-c38f5ca4-f751-4d8f-9ed4-885c7ceafad7 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 29 20:00:44.000137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:44.000137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 20:00:44.000137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 20:00:44.000137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 20:00:44.000137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 20:00:44.000137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 20:00:44.000137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 20:00:44.000137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 20:00:44.000137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application conn.query(q) Aug 29 20:00:44.000137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 20:00:44.000137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 20:00:44.000137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 20:00:44.000137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application result.read() Aug 29 20:00:44.000137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 20:00:44.001294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 20:00:44.001294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 20:00:44.001294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 20:00:44.001294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 20:00:44.001294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 20:00:44.001294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 20:00:44.001294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 20:00:44.001294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 20:00:44.001294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:44.001294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 20:00:44.001294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:44.001294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:44.001294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 20:00:44.001294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 20:00:44.001294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 29 20:00:44.001294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 29 20:00:44.001294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 20:00:44.001294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application next(self.gen) Aug 29 20:00:44.002720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 20:00:44.002720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application yield resource Aug 29 20:00:44.002720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 20:00:44.002720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application next(self.gen) Aug 29 20:00:44.002720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 20:00:44.002720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 20:00:44.002720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 20:00:44.002720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 20:00:44.002720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 20:00:44.002720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise self.value Aug 29 20:00:44.002720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 20:00:44.002720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 20:00:44.002720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 20:00:44.002720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application session.commit() Aug 29 20:00:44.002720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 20:00:44.002720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 20:00:44.002720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 20:00:44.004114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 20:00:44.004114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 20:00:44.004114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self.session.flush() Aug 29 20:00:44.004114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 20:00:44.004114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 20:00:44.004114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 20:00:44.004114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 20:00:44.004114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 20:00:44.004114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application compat.raise_( Aug 29 20:00:44.004114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 20:00:44.004114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception Aug 29 20:00:44.004114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 20:00:44.004114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 20:00:44.004114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 20:00:44.004114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 20:00:44.004114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 20:00:44.004114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 20:00:44.005030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 20:00:44.005030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 20:00:44.005030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 29 20:00:44.005030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 29 20:00:44.005030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 20:00:44.005030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 20:00:44.005030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 20:00:44.005030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 20:00:44.005030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 20:00:44.005030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 20:00:44.005030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 20:00:44.005030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 20:00:44.005030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 20:00:44.005030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 20:00:44.005030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 20:00:44.005030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception Aug 29 20:00:44.007083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 20:00:44.007083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 20:00:44.007083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 20:00:44.007083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 20:00:44.007083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 20:00:44.007083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 20:00:44.007083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 20:00:44.007083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application conn.query(q) Aug 29 20:00:44.007083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 20:00:44.007083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 20:00:44.007083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 20:00:44.007083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application result.read() Aug 29 20:00:44.007083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 20:00:44.007083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 20:00:44.007083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 20:00:44.007083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 20:00:44.009091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 20:00:44.009091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 20:00:44.009091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 20:00:44.009091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 20:00:44.009091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 20:00:44.009091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 29 20:00:44.009091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ee935ad3da4d46c9a89a942500d5a7e5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 29 20:00:44.009091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 20:00:44.009091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:44.009091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:44.009091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:44.009091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:44.009091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:44.009091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:44.009091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:44.011096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:44.011096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:44.011096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:44.011096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:44.011096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:44.011096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:44.011096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:44.011096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 29 20:00:44.011096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 29 20:00:44.011096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:44.011096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:44.011096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:44.011096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:44.011096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:44.011096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:44.011096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 29 20:00:44.011096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 29 20:00:44.013260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 29 20:00:44.013260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 29 20:00:44.013260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 20:00:44.013260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 20:00:44.013260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 29 20:00:44.013260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:44.013260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 487/963] 10.222.0.73 () {60 vars in 1131 bytes} [Mon Aug 29 20:00:43 2022] POST /identity/v3/users => generated 160 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:44.014790 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e720d20-5a40-4250-948d-4d6e055b1e5f None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:44.051039 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9896b3f-e518-4963-b28f-3e804a8437aa None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:44.051383 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9896b3f-e518-4963-b28f-3e804a8437aa None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:44.051666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9896b3f-e518-4963-b28f-3e804a8437aa None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:44.056324 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 488/964] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:00:44.062517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575b4a9-7379-40c0-8831-4f5ad94184ca None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:44.062804 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575b4a9-7379-40c0-8831-4f5ad94184ca None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:44.063082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575b4a9-7379-40c0-8831-4f5ad94184ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:44.085452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-e575b4a9-7379-40c0-8831-4f5ad94184ca None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:44.085452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-e575b4a9-7379-40c0-8831-4f5ad94184ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:44.125237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:44.139960 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e720d20-5a40-4250-948d-4d6e055b1e5f None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:44.167185 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 489/965] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:00:44.189208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0dd54-bfae-42be-bb2c-a3a470c27bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:44.189876 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0dd54-bfae-42be-bb2c-a3a470c27bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:44.190154 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0dd54-bfae-42be-bb2c-a3a470c27bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:44.190428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0dd54-bfae-42be-bb2c-a3a470c27bf3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:44.198215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0dd54-bfae-42be-bb2c-a3a470c27bf3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:44.199728 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0dd54-bfae-42be-bb2c-a3a470c27bf3 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:44.212706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-3bf0dd54-bfae-42be-bb2c-a3a470c27bf3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 20:00:44.212706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:44.212706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:44.212706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:44.212706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:44.212706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:44.212706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:44.212706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:44.212706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:44.212706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:44.212706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:44.212706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:44.212706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:00:44.212706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:00:44.212706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:00:44.212706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:00:44.213680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:44.213680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:44.213680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:44.213680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:44.213680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:44.213680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:44.213680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:44.213680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:44.213680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:44.213680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:44.213680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:44.213680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:44.213680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:44.213680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:44.213680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:44.213680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:44.213680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:44.213680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:44.214679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:44.214679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:44.214679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:00:44.214679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:00:44.214679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:00:44.214679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:00:44.214679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:00:44.214679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:00:44.214679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 20:00:44.214679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:44.214679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 477/966] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 20:00:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:44.220147 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575b4a9-7379-40c0-8831-4f5ad94184ca None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:44.328727 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575b4a9-7379-40c0-8831-4f5ad94184ca None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:44.364672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d516a-f123-41ee-81d3-a42f97b42768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:44.365197 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d516a-f123-41ee-81d3-a42f97b42768 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:44.365481 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d516a-f123-41ee-81d3-a42f97b42768 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:44.365772 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d516a-f123-41ee-81d3-a42f97b42768 None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:44.366480 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d516a-f123-41ee-81d3-a42f97b42768 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:44.366782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d516a-f123-41ee-81d3-a42f97b42768 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:44.368344 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d516a-f123-41ee-81d3-a42f97b42768 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:44.408127 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 490/967] 10.222.0.73 () {56 vars in 1101 bytes} [Mon Aug 29 20:00:44 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:44.444225 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0dd54-bfae-42be-bb2c-a3a470c27bf3 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:44.500474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:44.566672 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0dd54-bfae-42be-bb2c-a3a470c27bf3 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:44.603536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9510522-06bd-4304-934c-1910542c1fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:44.604268 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9510522-06bd-4304-934c-1910542c1fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:44.604552 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9510522-06bd-4304-934c-1910542c1fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:44.604828 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9510522-06bd-4304-934c-1910542c1fa9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:44.616711 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9510522-06bd-4304-934c-1910542c1fa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:00:44.622272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-d9510522-06bd-4304-934c-1910542c1fa9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:00:44.622272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:44.622272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:44.622272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:44.622272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:44.622272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:44.622272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:44.622272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:44.622272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:44.622272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:44.622272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:44.622272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:44.622272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:00:44.622272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:00:44.622272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:00:44.622272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:00:44.623188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:00:44.623188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:00:44.623188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:00:44.623188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:00:44.623188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:44.623188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:44.623188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:44.623188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:44.623188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:44.623188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:44.623188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:44.623188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:44.623188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:44.623188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:44.623188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:44.623188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:44.625620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:44.625620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:44.625620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:44.625620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:44.625620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:00:44.625620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:00:44.625620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:00:44.625620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:00:44.625620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:00:44.625620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:00:44.625620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:00:44.625620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:44.625620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 478/968] 10.222.0.73 () {56 vars in 1120 bytes} [Mon Aug 29 20:00:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:44.630406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d516a-f123-41ee-81d3-a42f97b42768 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:44.665115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:44.742029 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d516a-f123-41ee-81d3-a42f97b42768 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:44.774797 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1c8d5-89dd-4f23-91de-135521186377 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:44.775109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1c8d5-89dd-4f23-91de-135521186377 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:44.775339 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1c8d5-89dd-4f23-91de-135521186377 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:44.777736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 479/969] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:44.781186 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d063c-f0e3-4cfb-bf71-c8a1f62a7889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:44.781845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d063c-f0e3-4cfb-bf71-c8a1f62a7889 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:44.782037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd49213-de84-4572-a9c0-79508897e65a None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:44.782137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d063c-f0e3-4cfb-bf71-c8a1f62a7889 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:44.782324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd49213-de84-4572-a9c0-79508897e65a None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:44.782521 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d063c-f0e3-4cfb-bf71-c8a1f62a7889 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:44.782626 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd49213-de84-4572-a9c0-79508897e65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:44.783203 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d063c-f0e3-4cfb-bf71-c8a1f62a7889 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:44.783511 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d063c-f0e3-4cfb-bf71-c8a1f62a7889 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:44.784896 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d063c-f0e3-4cfb-bf71-c8a1f62a7889 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:44.794021 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 491/970] 10.222.0.73 () {56 vars in 1115 bytes} [Mon Aug 29 20:00:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:44.800230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d063c-f0e3-4cfb-bf71-c8a1f62a7889 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:44.802637 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-0dd49213-de84-4572-a9c0-79508897e65a None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:44.805049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-0dd49213-de84-4572-a9c0-79508897e65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:44.884773 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 480/971] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:00:44.908317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:44.918086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d063c-f0e3-4cfb-bf71-c8a1f62a7889 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:44.955015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ffd90c-c3c9-4514-a79f-def219fc4bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:44.955637 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffd90c-c3c9-4514-a79f-def219fc4bae None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:44.955901 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffd90c-c3c9-4514-a79f-def219fc4bae None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:44.956172 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffd90c-c3c9-4514-a79f-def219fc4bae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:44.967052 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ffd90c-c3c9-4514-a79f-def219fc4bae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:44.968615 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ffd90c-c3c9-4514-a79f-def219fc4bae None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:44.984724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-72ffd90c-c3c9-4514-a79f-def219fc4bae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:44.984724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:44.984724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:44.984724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:44.984724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:44.984724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:44.984724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:44.984724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:44.984724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:44.984724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:44.984724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:44.984724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:44.984724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:44.984724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:44.984724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:44.984724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:44.985784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:44.985784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:44.985784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:44.985784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:44.985784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:44.985784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:44.985784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:44.985784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:44.985784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:44.985784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:44.985784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:44.985784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:44.985784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:44.985784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:44.985784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:44.985784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:44.985784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:44.990732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:44.990732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:44.990732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:44.990732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 492/972] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:00:44.993086 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd49213-de84-4572-a9c0-79508897e65a None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:45.028518 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:45.108790 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd49213-de84-4572-a9c0-79508897e65a None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:45.144944 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098d83c-46f5-4b87-99fd-f6807865747b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:45.145587 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098d83c-46f5-4b87-99fd-f6807865747b None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:45.145835 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098d83c-46f5-4b87-99fd-f6807865747b None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:45.146034 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098d83c-46f5-4b87-99fd-f6807865747b None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:45.146679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098d83c-46f5-4b87-99fd-f6807865747b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:45.146972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098d83c-46f5-4b87-99fd-f6807865747b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:45.148337 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098d83c-46f5-4b87-99fd-f6807865747b None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:45.161157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 481/973] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:45.166586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ffd90c-c3c9-4514-a79f-def219fc4bae None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:45.166858 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098d83c-46f5-4b87-99fd-f6807865747b None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:45.281099 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:45.281961 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ffd90c-c3c9-4514-a79f-def219fc4bae None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:45.288890 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098d83c-46f5-4b87-99fd-f6807865747b None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:45.319142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693275ce-1196-44fe-9ecb-95b2aba0458c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:45.319775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693275ce-1196-44fe-9ecb-95b2aba0458c None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:45.320055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693275ce-1196-44fe-9ecb-95b2aba0458c None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:45.320329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693275ce-1196-44fe-9ecb-95b2aba0458c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:45.324169 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d3f63-802f-45e4-a49e-e4eb76e851e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:45.324797 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d3f63-802f-45e4-a49e-e4eb76e851e1 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:45.325076 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d3f63-802f-45e4-a49e-e4eb76e851e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:45.325349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d3f63-802f-45e4-a49e-e4eb76e851e1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:45.325960 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d3f63-802f-45e4-a49e-e4eb76e851e1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:45.327783 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d3f63-802f-45e4-a49e-e4eb76e851e1 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:45.328011 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693275ce-1196-44fe-9ecb-95b2aba0458c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:45.329428 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693275ce-1196-44fe-9ecb-95b2aba0458c None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:45.336720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-7a4d3f63-802f-45e4-a49e-e4eb76e851e1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:00:45.336720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:45.336720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:00:45.336720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:00:45.336720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:00:45.336720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:00:45.336720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:00:45.336720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:45.336720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:00:45.336720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:45.336720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:45.336720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:45.336720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:45.336720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:45.336720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:45.336720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:45.336720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:45.337929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:45.337929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:45.337929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:45.337929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:45.337929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:45.337929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:45.337929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:00:45.337929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:00:45.337929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:00:45.337929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:00:45.337929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:45.337929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:45.337929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:45.337929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:45.337929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:45.337929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:45.338781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:45.338781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:45.338781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:45.338781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:45.338781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:45.338781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:45.338781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:45.338781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:45.338781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:45.338781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:45.338781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:00:45.338781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:45.338781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:00:45.338781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:45.338781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 482/974] 10.222.0.73 () {56 vars in 1126 bytes} [Mon Aug 29 20:00:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:00:45.339561 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516692d5-3ece-4b09-a0a5-f7e801510194 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:45.339561 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516692d5-3ece-4b09-a0a5-f7e801510194 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:45.340054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516692d5-3ece-4b09-a0a5-f7e801510194 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:45.341001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-693275ce-1196-44fe-9ecb-95b2aba0458c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:00:45.341001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:45.341001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:45.341001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:45.341001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:45.341001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:45.341001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:45.341001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:45.341001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:45.341001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:45.341001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:45.341001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:45.341001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:00:45.341001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:00:45.341001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:00:45.341001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:00:45.343157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:45.343157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:45.343157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:45.343157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:45.343157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:45.343157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:45.343157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:45.343157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:45.343157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:45.343157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:45.343157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:45.343157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:45.343157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:45.343157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:45.343157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:45.343157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:45.343157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:45.343157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:45.344110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:45.344110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:45.344110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:00:45.344110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:00:45.344110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:00:45.344110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:00:45.344110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:00:45.344110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:00:45.344110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:00:45.344110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:45.344672 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 483/975] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:45.345986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 493/976] 10.222.0.73 () {56 vars in 1117 bytes} [Mon Aug 29 20:00:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:45.346889 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516692d5-3ece-4b09-a0a5-f7e801510194 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:45.348525 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693275ce-1196-44fe-9ecb-95b2aba0458c None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:45.352158 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:45.449755 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693275ce-1196-44fe-9ecb-95b2aba0458c None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:45.452784 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516692d5-3ece-4b09-a0a5-f7e801510194 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:45.485575 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e478fc-accd-46a8-ab10-6230cf18c6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:45.486176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e478fc-accd-46a8-ab10-6230cf18c6bc None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:45.486459 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e478fc-accd-46a8-ab10-6230cf18c6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:45.486728 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e478fc-accd-46a8-ab10-6230cf18c6bc None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:45.486965 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f49191-72a7-4005-9055-e16525dc4f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:45.487457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e478fc-accd-46a8-ab10-6230cf18c6bc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:45.487552 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f49191-72a7-4005-9055-e16525dc4f79 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:45.487762 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e478fc-accd-46a8-ab10-6230cf18c6bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:45.487850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f49191-72a7-4005-9055-e16525dc4f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:45.488098 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f49191-72a7-4005-9055-e16525dc4f79 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:45.488727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f49191-72a7-4005-9055-e16525dc4f79 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:45.489033 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f49191-72a7-4005-9055-e16525dc4f79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:45.489196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e478fc-accd-46a8-ab10-6230cf18c6bc None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:45.490403 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f49191-72a7-4005-9055-e16525dc4f79 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:45.495659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 484/977] 10.222.0.73 () {56 vars in 1122 bytes} [Mon Aug 29 20:00:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:45.496791 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801eb59-cddb-4a1f-b6ac-5737a6ef5937 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:45.497081 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801eb59-cddb-4a1f-b6ac-5737a6ef5937 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:45.497360 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801eb59-cddb-4a1f-b6ac-5737a6ef5937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:45.507746 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 494/978] 10.222.0.73 () {56 vars in 1149 bytes} [Mon Aug 29 20:00:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:45.508577 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e478fc-accd-46a8-ab10-6230cf18c6bc None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:45.517683 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-d801eb59-cddb-4a1f-b6ac-5737a6ef5937 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:45.518931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d801eb59-cddb-4a1f-b6ac-5737a6ef5937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:45.536294 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:45.574185 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:45.587725 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 485/979] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:45.588568 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801eb59-cddb-4a1f-b6ac-5737a6ef5937 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:45.619698 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e478fc-accd-46a8-ab10-6230cf18c6bc None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:45.660070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543e529-bcf9-419c-9e1a-4462d08e8a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:45.660833 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543e529-bcf9-419c-9e1a-4462d08e8a0c None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:45.661113 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543e529-bcf9-419c-9e1a-4462d08e8a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:45.661388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543e529-bcf9-419c-9e1a-4462d08e8a0c None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:45.662055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543e529-bcf9-419c-9e1a-4462d08e8a0c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:45.662362 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543e529-bcf9-419c-9e1a-4462d08e8a0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:45.663739 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543e529-bcf9-419c-9e1a-4462d08e8a0c None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:45.668583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 495/980] 10.222.0.73 () {56 vars in 1122 bytes} [Mon Aug 29 20:00:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:45.674452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543e529-bcf9-419c-9e1a-4462d08e8a0c None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:45.712433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801eb59-cddb-4a1f-b6ac-5737a6ef5937 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:45.749800 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e48e52-d953-4146-afc7-cae14030d105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:45.750655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e48e52-d953-4146-afc7-cae14030d105 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:45.751080 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e48e52-d953-4146-afc7-cae14030d105 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:45.751744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e48e52-d953-4146-afc7-cae14030d105 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:45.775278 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:45.781084 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543e529-bcf9-419c-9e1a-4462d08e8a0c None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:45.807249 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:45.817642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b4986-3208-4709-b0e2-61b084828c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:45.818275 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b4986-3208-4709-b0e2-61b084828c04 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:45.818553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b4986-3208-4709-b0e2-61b084828c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:45.818823 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b4986-3208-4709-b0e2-61b084828c04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:45.819522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b4986-3208-4709-b0e2-61b084828c04 None admin] RBAC: Adding query filter params (interface=admin, service_id=306bd2e233cc4dc185826aee41f43c3f, region_id=RegionOne) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:45.819825 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b4986-3208-4709-b0e2-61b084828c04 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:45.821185 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b4986-3208-4709-b0e2-61b084828c04 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:45.826376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 496/981] 10.222.0.73 () {56 vars in 1249 bytes} [Mon Aug 29 20:00:45 2022] GET /identity/v3/endpoints?service_id=306bd2e233cc4dc185826aee41f43c3f&interface=admin®ion_id=RegionOne => generated 193 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:45.830904 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e48e52-d953-4146-afc7-cae14030d105 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:45.832548 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b4986-3208-4709-b0e2-61b084828c04 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:45.832679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e48e52-d953-4146-afc7-cae14030d105 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:45.934664 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b4986-3208-4709-b0e2-61b084828c04 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:45.939067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 486/982] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:00:45 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:00:45.939880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e48e52-d953-4146-afc7-cae14030d105 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:45.970444 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4919f-8576-4dc5-b0f8-e3488920fc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:45.971035 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4919f-8576-4dc5-b0f8-e3488920fc38 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:45.971326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4919f-8576-4dc5-b0f8-e3488920fc38 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:45.971599 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4919f-8576-4dc5-b0f8-e3488920fc38 None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:45.972204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4919f-8576-4dc5-b0f8-e3488920fc38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:45.973564 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4919f-8576-4dc5-b0f8-e3488920fc38 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:45.979481 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 497/983] 10.222.0.73 () {56 vars in 1087 bytes} [Mon Aug 29 20:00:45 2022] GET /identity/v3/services => generated 1497 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:00:45.980260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4919f-8576-4dc5-b0f8-e3488920fc38 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:46.006717 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:46.049174 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:46.067093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e48e52-d953-4146-afc7-cae14030d105 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:46.114074 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576e5e9-6462-46de-aa06-153ba05ac1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:46.114074 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576e5e9-6462-46de-aa06-153ba05ac1d1 None admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:46.114074 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576e5e9-6462-46de-aa06-153ba05ac1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:46.114074 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576e5e9-6462-46de-aa06-153ba05ac1d1 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:46.115317 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c576e5e9-6462-46de-aa06-153ba05ac1d1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:46.115444 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4919f-8576-4dc5-b0f8-e3488920fc38 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:46.115671 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c576e5e9-6462-46de-aa06-153ba05ac1d1 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:46.132970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 487/984] 10.222.0.73 () {60 vars in 1141 bytes} [Mon Aug 29 20:00:45 2022] POST /identity/v3/services => generated 242 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:00:46.176837 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215bf13-7f6d-44ac-9e86-655269aaba24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:46.177489 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215bf13-7f6d-44ac-9e86-655269aaba24 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:46.177815 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215bf13-7f6d-44ac-9e86-655269aaba24 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:46.178087 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215bf13-7f6d-44ac-9e86-655269aaba24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:46.192127 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1215bf13-7f6d-44ac-9e86-655269aaba24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:46.193597 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1215bf13-7f6d-44ac-9e86-655269aaba24 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:46.210808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-1215bf13-7f6d-44ac-9e86-655269aaba24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:46.210808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:46.210808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:46.210808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:46.210808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:46.210808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:46.210808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:46.210808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:46.210808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:46.210808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:46.210808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:46.210808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:46.210808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:46.210808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:46.210808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:46.210808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:46.211983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:46.211983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:46.211983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:46.211983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:46.211983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:46.211983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:46.211983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:46.211983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:46.211983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:46.211983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:46.211983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:46.211983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:46.211983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:46.211983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:46.211983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:46.211983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:46.211983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:46.212989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:46.212989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:46.212989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:46.213814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 498/985] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:46.218662 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576e5e9-6462-46de-aa06-153ba05ac1d1 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:46.348937 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576e5e9-6462-46de-aa06-153ba05ac1d1 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:46.365797 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:46.394637 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ac80e-73e5-4dda-adc2-1bec4acc87eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:46.395340 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ac80e-73e5-4dda-adc2-1bec4acc87eb None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:46.395709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ac80e-73e5-4dda-adc2-1bec4acc87eb None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:46.395973 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ac80e-73e5-4dda-adc2-1bec4acc87eb None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:46.396623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ac80e-73e5-4dda-adc2-1bec4acc87eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:46.396923 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ac80e-73e5-4dda-adc2-1bec4acc87eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:46.398317 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ac80e-73e5-4dda-adc2-1bec4acc87eb None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:46.412635 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 488/986] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:46.418342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215bf13-7f6d-44ac-9e86-655269aaba24 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:46.526724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215bf13-7f6d-44ac-9e86-655269aaba24 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:46.528565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:46.565119 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1261d-9a9f-43b5-8004-e6a5e4248f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:46.566145 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1261d-9a9f-43b5-8004-e6a5e4248f1b None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:46.566631 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1261d-9a9f-43b5-8004-e6a5e4248f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:46.567130 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1261d-9a9f-43b5-8004-e6a5e4248f1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:46.574393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1261d-9a9f-43b5-8004-e6a5e4248f1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:46.576057 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1261d-9a9f-43b5-8004-e6a5e4248f1b None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:46.587876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-86c1261d-9a9f-43b5-8004-e6a5e4248f1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:46.587876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:46.587876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:46.587876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:46.587876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:46.587876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:46.587876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:46.587876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:46.587876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:46.587876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:46.587876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:46.587876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:46.587876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:46.587876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:46.587876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:46.588780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:46.588780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:46.588780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:46.588780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:46.588780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:46.588780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:46.588780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:46.588780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:46.588780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:46.588780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:46.588780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:46.588780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:46.588780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:46.588780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:46.588780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:46.588780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:46.588780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:46.589680 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:46.589680 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:46.589680 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:46.589680 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:46.589680 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:46.589680 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:46.589680 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:46.589680 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:46.596504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 499/987] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:46.598156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ac80e-73e5-4dda-adc2-1bec4acc87eb None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:46.727493 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ac80e-73e5-4dda-adc2-1bec4acc87eb None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:46.766023 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee136e5e-f8d7-477b-b651-c91d4c245ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:46.766937 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee136e5e-f8d7-477b-b651-c91d4c245ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:46.767422 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee136e5e-f8d7-477b-b651-c91d4c245ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:46.767892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee136e5e-f8d7-477b-b651-c91d4c245ca3 None admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:46.768761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee136e5e-f8d7-477b-b651-c91d4c245ca3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:46.769237 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee136e5e-f8d7-477b-b651-c91d4c245ca3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:46.770878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee136e5e-f8d7-477b-b651-c91d4c245ca3 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:46.786223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 489/988] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:00:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:46.791861 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1261d-9a9f-43b5-8004-e6a5e4248f1b None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:46.935663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1261d-9a9f-43b5-8004-e6a5e4248f1b None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:46.995623 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5940e-2041-41cf-9746-2c7b61fe6de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:46.997802 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5940e-2041-41cf-9746-2c7b61fe6de9 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:46.998537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5940e-2041-41cf-9746-2c7b61fe6de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:46.999643 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5940e-2041-41cf-9746-2c7b61fe6de9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:47.001616 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5940e-2041-41cf-9746-2c7b61fe6de9 None admin] RBAC: Adding query filter params (role.id=64c60e8079b94f7b9cf2473b4b75f5a5, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=e0c76478b6694782b896cdf9cda32d9c) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:47.002347 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5940e-2041-41cf-9746-2c7b61fe6de9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:47.005160 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5940e-2041-41cf-9746-2c7b61fe6de9 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:47.018301 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 500/989] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:00:46 2022] GET /identity/v3/role_assignments?user.id=e0c76478b6694782b896cdf9cda32d9c&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=64c60e8079b94f7b9cf2473b4b75f5a5 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:48.030868 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf5d72-df19-4860-b48f-7ae4098ebbbe None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:48.031378 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf5d72-df19-4860-b48f-7ae4098ebbbe None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:48.031482 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf5d72-df19-4860-b48f-7ae4098ebbbe None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:48.033501 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 490/990] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:48.038667 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b680a2-685c-44a4-a871-81250e79330c None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:48.039104 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b680a2-685c-44a4-a871-81250e79330c None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:48.039273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b680a2-685c-44a4-a871-81250e79330c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:48.058229 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-56b680a2-685c-44a4-a871-81250e79330c None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:48.059429 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-56b680a2-685c-44a4-a871-81250e79330c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:48.088680 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:48.118747 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9fedb-b89f-4b14-8f1f-60531af4fa6d None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:48.119134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9fedb-b89f-4b14-8f1f-60531af4fa6d None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:48.119476 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9fedb-b89f-4b14-8f1f-60531af4fa6d None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:48.121030 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 491/991] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:48.126641 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022a362-50c0-4bce-89ee-fd6b424a8603 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:48.127142 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022a362-50c0-4bce-89ee-fd6b424a8603 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:48.127576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022a362-50c0-4bce-89ee-fd6b424a8603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:48.142137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 501/992] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:48.146021 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-e022a362-50c0-4bce-89ee-fd6b424a8603 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:48.147146 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-e022a362-50c0-4bce-89ee-fd6b424a8603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:48.176972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:48.220312 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 492/993] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:00:48.415160 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b680a2-685c-44a4-a871-81250e79330c None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:48.491361 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022a362-50c0-4bce-89ee-fd6b424a8603 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:48.521006 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b680a2-685c-44a4-a871-81250e79330c None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:48.557752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb50996-4daa-4d23-b0c4-4c50036db327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:48.558388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb50996-4daa-4d23-b0c4-4c50036db327 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:48.558689 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb50996-4daa-4d23-b0c4-4c50036db327 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:48.558932 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb50996-4daa-4d23-b0c4-4c50036db327 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:48.559518 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb50996-4daa-4d23-b0c4-4c50036db327 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:48.560885 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb50996-4daa-4d23-b0c4-4c50036db327 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:48.571633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-3fb50996-4daa-4d23-b0c4-4c50036db327 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:00:48.571633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:48.571633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:00:48.571633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:00:48.571633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:00:48.571633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:00:48.571633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:00:48.571633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:48.571633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:00:48.571633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:48.571633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:48.571633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:48.571633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:48.571633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:48.571633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:48.571633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:48.571633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:48.572676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:48.572676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:48.572676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:48.572676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:48.572676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:48.572676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:48.572676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:00:48.572676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:00:48.572676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:00:48.572676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:00:48.572676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:48.572676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:48.572676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:48.572676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:48.572676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:48.572676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:48.573516 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:48.573516 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:48.573516 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:48.573516 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:48.573516 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:48.573516 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:48.573516 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:48.573516 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:48.573516 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:48.573516 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:48.573516 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:00:48.573516 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:48.573516 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:00:48.573516 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:48.577881 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 502/994] 10.222.0.73 () {56 vars in 1126 bytes} [Mon Aug 29 20:00:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:48.579237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb50996-4daa-4d23-b0c4-4c50036db327 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:48.599286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022a362-50c0-4bce-89ee-fd6b424a8603 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:48.604870 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:48.637679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1864d1-1aa7-481d-bf33-9230717e9527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:48.638452 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1864d1-1aa7-481d-bf33-9230717e9527 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:48.638734 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1864d1-1aa7-481d-bf33-9230717e9527 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:48.639000 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1864d1-1aa7-481d-bf33-9230717e9527 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:48.639607 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1864d1-1aa7-481d-bf33-9230717e9527 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:48.641127 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1864d1-1aa7-481d-bf33-9230717e9527 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:48.650975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-9b1864d1-1aa7-481d-bf33-9230717e9527 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:00:48.650975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:48.650975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:00:48.650975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:00:48.650975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:00:48.650975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:00:48.650975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:00:48.650975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:48.650975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:00:48.650975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:48.650975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:48.650975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:48.650975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:48.650975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:48.650975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:48.650975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:48.650975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:48.652015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:48.652015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:48.652015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:48.652015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:48.652015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:48.652015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:48.652015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:00:48.652015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:00:48.652015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:00:48.652015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:00:48.652015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:48.652015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:48.652015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:48.652015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:48.652015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:48.652015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:48.652880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:48.652880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:48.652880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:48.652880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:48.652880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:48.652880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:48.652880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:48.652880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:48.652880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:48.652880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:48.652880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:00:48.652880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:48.652880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:00:48.652880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:48.652880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 493/995] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:00:48.658035 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1864d1-1aa7-481d-bf33-9230717e9527 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:48.689619 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:48.695581 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb50996-4daa-4d23-b0c4-4c50036db327 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:48.732744 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626c1d6-1959-41a0-b6d5-f37b1c18c08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:48.733915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626c1d6-1959-41a0-b6d5-f37b1c18c08d None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:48.734194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626c1d6-1959-41a0-b6d5-f37b1c18c08d None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:48.734462 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626c1d6-1959-41a0-b6d5-f37b1c18c08d None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:48.735083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626c1d6-1959-41a0-b6d5-f37b1c18c08d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:48.735383 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626c1d6-1959-41a0-b6d5-f37b1c18c08d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:48.736760 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626c1d6-1959-41a0-b6d5-f37b1c18c08d None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:48.746560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 503/996] 10.222.0.73 () {56 vars in 1122 bytes} [Mon Aug 29 20:00:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:48.747671 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626c1d6-1959-41a0-b6d5-f37b1c18c08d None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:48.772440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1864d1-1aa7-481d-bf33-9230717e9527 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:48.809179 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1aa77f-1fbe-4c44-8cb1-1e443833f8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:48.809873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1aa77f-1fbe-4c44-8cb1-1e443833f8e7 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:48.810264 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1aa77f-1fbe-4c44-8cb1-1e443833f8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:48.810534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1aa77f-1fbe-4c44-8cb1-1e443833f8e7 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:48.811269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1aa77f-1fbe-4c44-8cb1-1e443833f8e7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:48.811568 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1aa77f-1fbe-4c44-8cb1-1e443833f8e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:48.813587 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1aa77f-1fbe-4c44-8cb1-1e443833f8e7 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:48.823603 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 494/997] 10.222.0.73 () {56 vars in 1112 bytes} [Mon Aug 29 20:00:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:48.824447 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1aa77f-1fbe-4c44-8cb1-1e443833f8e7 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:48.855557 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:48.884578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626c1d6-1959-41a0-b6d5-f37b1c18c08d None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:48.928788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:48.934486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67cbcd-8844-4dc1-8e71-1fd0d8e4b077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:48.935113 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67cbcd-8844-4dc1-8e71-1fd0d8e4b077 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:48.935527 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67cbcd-8844-4dc1-8e71-1fd0d8e4b077 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:48.935802 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67cbcd-8844-4dc1-8e71-1fd0d8e4b077 None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:48.936764 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c67cbcd-8844-4dc1-8e71-1fd0d8e4b077 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:48.937341 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c67cbcd-8844-4dc1-8e71-1fd0d8e4b077 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:48.938751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c67cbcd-8844-4dc1-8e71-1fd0d8e4b077 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:48.938958 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1aa77f-1fbe-4c44-8cb1-1e443833f8e7 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:48.944859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 504/998] 10.222.0.73 () {56 vars in 1122 bytes} [Mon Aug 29 20:00:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:48.950077 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67cbcd-8844-4dc1-8e71-1fd0d8e4b077 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:48.977845 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aba2f4-9396-4684-adc4-22f1cb3163cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:48.978461 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aba2f4-9396-4684-adc4-22f1cb3163cf None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:48.978742 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aba2f4-9396-4684-adc4-22f1cb3163cf None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:48.979018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aba2f4-9396-4684-adc4-22f1cb3163cf None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:48.979688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aba2f4-9396-4684-adc4-22f1cb3163cf None admin] RBAC: Adding query filter params (type=compute) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:48.979992 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aba2f4-9396-4684-adc4-22f1cb3163cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:48.981359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aba2f4-9396-4684-adc4-22f1cb3163cf None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:48.987385 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 495/999] 10.222.0.73 () {56 vars in 1112 bytes} [Mon Aug 29 20:00:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:48.994672 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aba2f4-9396-4684-adc4-22f1cb3163cf None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:49.067029 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67cbcd-8844-4dc1-8e71-1fd0d8e4b077 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:49.103482 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aba2f4-9396-4684-adc4-22f1cb3163cf None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:49.105085 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cacd9c7-3cc8-4739-808f-1cc3377b9eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:49.105698 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacd9c7-3cc8-4739-808f-1cc3377b9eb2 None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:49.106011 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacd9c7-3cc8-4739-808f-1cc3377b9eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:49.106283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacd9c7-3cc8-4739-808f-1cc3377b9eb2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:49.106952 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cacd9c7-3cc8-4739-808f-1cc3377b9eb2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:49.108959 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cacd9c7-3cc8-4739-808f-1cc3377b9eb2 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:49.125466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 505/1000] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:00:48 2022] POST /identity/v3/endpoints => generated 325 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:00:49.127024 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ab522-482b-444e-bc66-6c2866a4173b None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:49.127364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ab522-482b-444e-bc66-6c2866a4173b None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:49.127643 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ab522-482b-444e-bc66-6c2866a4173b None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:49.129857 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 506/1001] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:49.134794 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e333fe4-851c-426f-88bb-bcf07f6af3cf None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:49.135083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e333fe4-851c-426f-88bb-bcf07f6af3cf None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:49.135365 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e333fe4-851c-426f-88bb-bcf07f6af3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:49.150996 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f896d-c7d7-4b36-a1f5-cd3f15f534bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:49.151646 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f896d-c7d7-4b36-a1f5-cd3f15f534bf None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:49.151915 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f896d-c7d7-4b36-a1f5-cd3f15f534bf None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:49.152191 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f896d-c7d7-4b36-a1f5-cd3f15f534bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:49.152988 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f896d-c7d7-4b36-a1f5-cd3f15f534bf None admin] RBAC: Adding query filter params (interface=public, service_id=c9ca0e9048774b528f23a792e7e361d5, region_id=RegionOne) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:49.153300 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f896d-c7d7-4b36-a1f5-cd3f15f534bf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:49.154749 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f896d-c7d7-4b36-a1f5-cd3f15f534bf None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:49.156266 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-7e333fe4-851c-426f-88bb-bcf07f6af3cf None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:49.157532 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-7e333fe4-851c-426f-88bb-bcf07f6af3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:49.160117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:49.162917 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 496/1002] 10.222.0.73 () {56 vars in 1251 bytes} [Mon Aug 29 20:00:48 2022] GET /identity/v3/endpoints?service_id=c9ca0e9048774b528f23a792e7e361d5&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:49.168284 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f896d-c7d7-4b36-a1f5-cd3f15f534bf None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:49.191010 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:49.253829 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 507/1003] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:49 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:49.299348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f896d-c7d7-4b36-a1f5-cd3f15f534bf None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:49.335989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf50763-63e0-41f9-94b0-0c9350c73508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:49.336566 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf50763-63e0-41f9-94b0-0c9350c73508 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:49.336786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf50763-63e0-41f9-94b0-0c9350c73508 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:49.337057 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf50763-63e0-41f9-94b0-0c9350c73508 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:49.337938 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf50763-63e0-41f9-94b0-0c9350c73508 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:49.339396 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf50763-63e0-41f9-94b0-0c9350c73508 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:49.347577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 497/1004] 10.222.0.73 () {56 vars in 1087 bytes} [Mon Aug 29 20:00:49 2022] GET /identity/v3/services => generated 1727 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:00:49.557171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e333fe4-851c-426f-88bb-bcf07f6af3cf None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:49.680856 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e333fe4-851c-426f-88bb-bcf07f6af3cf None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:49.718223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03fadd-4007-4b29-a008-ca5f2deed93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:49.718864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03fadd-4007-4b29-a008-ca5f2deed93d None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:49.719143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03fadd-4007-4b29-a008-ca5f2deed93d None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:49.719416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03fadd-4007-4b29-a008-ca5f2deed93d None admin] PATH_INFO: `/v3/services/image` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:49.719998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03fadd-4007-4b29-a008-ca5f2deed93d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:49.721374 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03fadd-4007-4b29-a008-ca5f2deed93d None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:49.730470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-bb03fadd-4007-4b29-a008-ca5f2deed93d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:00:49.730470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:49.730470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:00:49.730470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:00:49.730470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:00:49.730470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:00:49.730470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:00:49.730470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:49.730470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:00:49.730470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:49.730470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:49.730470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:49.730470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:49.730470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:49.730470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:49.730470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:49.730470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:49.731636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:49.731636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:49.731636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:49.731636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:49.731636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:49.731636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:49.731636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:00:49.731636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:00:49.731636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:00:49.731636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:00:49.731636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:49.731636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:49.731636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:49.731636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:49.731636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:49.731636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:49.732588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:49.732588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:49.732588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:49.732588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:49.732588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:49.732588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:49.732588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:49.732588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:49.732588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:49.732588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:49.732588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:00:49.732588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:49.732588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:00:49.732588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:49.732588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 508/1005] 10.222.0.73 () {56 vars in 1105 bytes} [Mon Aug 29 20:00:49 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:49.737647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf50763-63e0-41f9-94b0-0c9350c73508 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:49.821942 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:49.845543 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf50763-63e0-41f9-94b0-0c9350c73508 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:49.887864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017dcbc4-2805-4c7c-8a6a-bee20d364d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:49.888472 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017dcbc4-2805-4c7c-8a6a-bee20d364d7e None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:49.888752 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017dcbc4-2805-4c7c-8a6a-bee20d364d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:49.889024 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017dcbc4-2805-4c7c-8a6a-bee20d364d7e None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:49.890362 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017dcbc4-2805-4c7c-8a6a-bee20d364d7e None admin] RBAC: Adding query filter params (name=image) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:49.890659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017dcbc4-2805-4c7c-8a6a-bee20d364d7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:49.892026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017dcbc4-2805-4c7c-8a6a-bee20d364d7e None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:49.901389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 498/1006] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:49.903032 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03fadd-4007-4b29-a008-ca5f2deed93d None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:49.906177 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:50.009875 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03fadd-4007-4b29-a008-ca5f2deed93d None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:50.043374 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b3784-310c-48f9-bcc6-b5f43a59d596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:50.043992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b3784-310c-48f9-bcc6-b5f43a59d596 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:50.044371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b3784-310c-48f9-bcc6-b5f43a59d596 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:50.044546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b3784-310c-48f9-bcc6-b5f43a59d596 None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:50.045169 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b3784-310c-48f9-bcc6-b5f43a59d596 None admin] RBAC: Adding query filter params (type=image) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:50.045474 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b3784-310c-48f9-bcc6-b5f43a59d596 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:50.047566 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b3784-310c-48f9-bcc6-b5f43a59d596 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:50.057783 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 509/1007] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:49 2022] GET /identity/v3/services?type=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:51.142733 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018febd-8fa8-43d6-a5f8-307afe551fa0 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:51.143245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018febd-8fa8-43d6-a5f8-307afe551fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:51.143324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018febd-8fa8-43d6-a5f8-307afe551fa0 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:51.144640 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 499/1008] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:51.150243 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8407dc-57d4-4a01-b5f2-796b4dc4dd0b None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:51.150734 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8407dc-57d4-4a01-b5f2-796b4dc4dd0b None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:51.150814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8407dc-57d4-4a01-b5f2-796b4dc4dd0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:51.171385 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-ba8407dc-57d4-4a01-b5f2-796b4dc4dd0b None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:51.172615 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ba8407dc-57d4-4a01-b5f2-796b4dc4dd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:51.236668 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:51.249753 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 510/1009] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:51 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:51.310006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5cfc3-d971-4c61-8d9f-c2db259d2b4b None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:51.310294 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5cfc3-d971-4c61-8d9f-c2db259d2b4b None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:51.310746 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5cfc3-d971-4c61-8d9f-c2db259d2b4b None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:51.312140 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 500/1010] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:51.317438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1856d8-d2c3-4521-bee9-b67366a66cb9 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:51.317751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1856d8-d2c3-4521-bee9-b67366a66cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:51.318025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1856d8-d2c3-4521-bee9-b67366a66cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:51.336432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-8c1856d8-d2c3-4521-bee9-b67366a66cb9 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:51.337685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-8c1856d8-d2c3-4521-bee9-b67366a66cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:51.411131 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 511/1011] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:51 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:51.524246 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5cfc3-d971-4c61-8d9f-c2db259d2b4b None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:51.632904 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5cfc3-d971-4c61-8d9f-c2db259d2b4b None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:51.668902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fc1d5-f35c-4d08-a076-be0ad60d587e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:51.669539 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fc1d5-f35c-4d08-a076-be0ad60d587e None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:51.669849 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fc1d5-f35c-4d08-a076-be0ad60d587e None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:51.670123 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fc1d5-f35c-4d08-a076-be0ad60d587e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:51.671265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5fc1d5-f35c-4d08-a076-be0ad60d587e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:51.672692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5fc1d5-f35c-4d08-a076-be0ad60d587e None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:51.674480 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 501/1012] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:51.680151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1856d8-d2c3-4521-bee9-b67366a66cb9 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:51.697366 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fc1d5-f35c-4d08-a076-be0ad60d587e None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:51.710819 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:51.790665 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1856d8-d2c3-4521-bee9-b67366a66cb9 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:51.806660 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fc1d5-f35c-4d08-a076-be0ad60d587e None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:51.820477 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:51.832797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df311344-98ca-417e-a3f2-9df0626feda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:51.833416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df311344-98ca-417e-a3f2-9df0626feda6 None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:51.833696 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df311344-98ca-417e-a3f2-9df0626feda6 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:51.834007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df311344-98ca-417e-a3f2-9df0626feda6 None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:51.834679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df311344-98ca-417e-a3f2-9df0626feda6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:51.836121 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df311344-98ca-417e-a3f2-9df0626feda6 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:51.843800 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b42c26-995a-4328-85d6-66b2cf10d8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:51.844450 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b42c26-995a-4328-85d6-66b2cf10d8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:51.844729 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b42c26-995a-4328-85d6-66b2cf10d8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:51.845000 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b42c26-995a-4328-85d6-66b2cf10d8a3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:51.845587 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b42c26-995a-4328-85d6-66b2cf10d8a3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:51.847090 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b42c26-995a-4328-85d6-66b2cf10d8a3 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:51.857618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-03b42c26-995a-4328-85d6-66b2cf10d8a3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:00:51.857618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:51.857618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:00:51.857618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:00:51.857618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:00:51.857618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:00:51.857618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:00:51.857618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:51.857618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:00:51.857618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:51.857618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:51.857618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:51.857618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:51.857618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:51.857618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:51.857618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:51.857618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:51.859042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:51.859042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:51.859042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:51.859042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:51.859042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:51.859042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:51.859042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:00:51.859042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:00:51.859042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:00:51.859042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:00:51.859042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:51.859042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:51.859042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:51.859042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:51.859042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:51.859042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:51.860016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:51.860016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:51.860016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:51.860016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:51.860016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:51.860016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:51.860016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:51.860016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:51.860016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:51.860016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:51.860016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:00:51.860016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:51.860016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:00:51.860016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:51.864258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 502/1013] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:00:51.865182 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b42c26-995a-4328-85d6-66b2cf10d8a3 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:51.871010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 512/1014] 10.222.0.73 () {60 vars in 1141 bytes} [Mon Aug 29 20:00:51 2022] POST /identity/v3/projects => generated 311 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:00:51.953187 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:51.982402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b42c26-995a-4328-85d6-66b2cf10d8a3 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:52.005304 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6a02b-b2ad-4844-8ebc-3ddef2d9fa7e None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:52.005621 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6a02b-b2ad-4844-8ebc-3ddef2d9fa7e None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:52.005884 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6a02b-b2ad-4844-8ebc-3ddef2d9fa7e None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:52.007598 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 513/1015] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:00:52.012648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b9981-f0ec-4bc7-bc70-25d0658ef2b3 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:52.012927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b9981-f0ec-4bc7-bc70-25d0658ef2b3 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:52.013198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b9981-f0ec-4bc7-bc70-25d0658ef2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:52.024343 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deddbd0f-ce79-441f-9477-737857aedcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:52.024962 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddbd0f-ce79-441f-9477-737857aedcb3 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:52.025245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddbd0f-ce79-441f-9477-737857aedcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:52.025522 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddbd0f-ce79-441f-9477-737857aedcb3 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:52.026193 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deddbd0f-ce79-441f-9477-737857aedcb3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:52.027113 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deddbd0f-ce79-441f-9477-737857aedcb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:52.027902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deddbd0f-ce79-441f-9477-737857aedcb3 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:52.037736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 503/1016] 10.222.0.73 () {56 vars in 1112 bytes} [Mon Aug 29 20:00:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:52.037911 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-d08b9981-f0ec-4bc7-bc70-25d0658ef2b3 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:52.038496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-d08b9981-f0ec-4bc7-bc70-25d0658ef2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:52.039792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deddbd0f-ce79-441f-9477-737857aedcb3 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:52.119469 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 514/1017] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:52 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:00:52.150975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deddbd0f-ce79-441f-9477-737857aedcb3 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:52.187295 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c331a-0af1-4842-96bd-f19ce92c5d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:52.187902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c331a-0af1-4842-96bd-f19ce92c5d4a None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:52.188182 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c331a-0af1-4842-96bd-f19ce92c5d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:52.188456 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c331a-0af1-4842-96bd-f19ce92c5d4a None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:52.189084 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c331a-0af1-4842-96bd-f19ce92c5d4a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:52.189387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c331a-0af1-4842-96bd-f19ce92c5d4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:52.190787 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c331a-0af1-4842-96bd-f19ce92c5d4a None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:52.195661 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 504/1018] 10.222.0.73 () {56 vars in 1112 bytes} [Mon Aug 29 20:00:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:52.201264 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b9981-f0ec-4bc7-bc70-25d0658ef2b3 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:52.307379 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b9981-f0ec-4bc7-bc70-25d0658ef2b3 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:52.317880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:52.345827 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02395e2-7ed9-4995-9964-d74b916a9c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:52.346441 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02395e2-7ed9-4995-9964-d74b916a9c96 None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:52.346718 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02395e2-7ed9-4995-9964-d74b916a9c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:52.346989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02395e2-7ed9-4995-9964-d74b916a9c96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:52.347668 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02395e2-7ed9-4995-9964-d74b916a9c96 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:52.349043 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02395e2-7ed9-4995-9964-d74b916a9c96 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:52.372240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 515/1019] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:00:52 2022] POST /identity/v3/endpoints => generated 334 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:00:52.393767 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c331a-0af1-4842-96bd-f19ce92c5d4a None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:52.397184 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:52.516467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c331a-0af1-4842-96bd-f19ce92c5d4a None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:52.563313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68372318-897e-4796-8e16-d715b737a6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:52.563856 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68372318-897e-4796-8e16-d715b737a6f4 None admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:52.564137 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68372318-897e-4796-8e16-d715b737a6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:52.564411 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68372318-897e-4796-8e16-d715b737a6f4 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:52.565091 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68372318-897e-4796-8e16-d715b737a6f4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:52.566506 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68372318-897e-4796-8e16-d715b737a6f4 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:52.584345 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 505/1020] 10.222.0.73 () {60 vars in 1141 bytes} [Mon Aug 29 20:00:52 2022] POST /identity/v3/services => generated 242 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:00:53.893078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9d1b5-8d75-4a8b-b173-157615f5d24b None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:53.893956 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9d1b5-8d75-4a8b-b173-157615f5d24b None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:53.893956 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9d1b5-8d75-4a8b-b173-157615f5d24b None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:53.895032 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 516/1021] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:53.900322 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd9eff-a7cc-46a6-90c1-0062cedb9821 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:53.900871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd9eff-a7cc-46a6-90c1-0062cedb9821 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:53.901444 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd9eff-a7cc-46a6-90c1-0062cedb9821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:53.921970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-f7fd9eff-a7cc-46a6-90c1-0062cedb9821 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:53.923181 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-f7fd9eff-a7cc-46a6-90c1-0062cedb9821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:53.980496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:54.007858 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 506/1022] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:54.279513 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9d1b5-8d75-4a8b-b173-157615f5d24b None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:54.422143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9d1b5-8d75-4a8b-b173-157615f5d24b None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:54.456763 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa33b6-1aed-4b08-be68-144dc87b749f None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:54.457116 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa33b6-1aed-4b08-be68-144dc87b749f None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:54.457337 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa33b6-1aed-4b08-be68-144dc87b749f None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:54.458856 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 507/1023] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:00:54.463975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0811f9c-b6fe-4131-a253-138b6c57d26c None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:54.464343 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0811f9c-b6fe-4131-a253-138b6c57d26c None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:54.464581 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0811f9c-b6fe-4131-a253-138b6c57d26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:54.488162 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939dcf8-9a2a-4820-af0b-f52020cf11b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:54.488814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939dcf8-9a2a-4820-af0b-f52020cf11b6 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:54.489097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939dcf8-9a2a-4820-af0b-f52020cf11b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:54.489368 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939dcf8-9a2a-4820-af0b-f52020cf11b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:54.490577 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939dcf8-9a2a-4820-af0b-f52020cf11b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:54.492031 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939dcf8-9a2a-4820-af0b-f52020cf11b6 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:54.493750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 517/1024] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:54.493930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-d0811f9c-b6fe-4131-a253-138b6c57d26c None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:54.495988 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d0811f9c-b6fe-4131-a253-138b6c57d26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:54.501750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939dcf8-9a2a-4820-af0b-f52020cf11b6 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:54.540155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:54.584996 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 508/1025] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:00:54.586720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1236b4-090f-4b6c-8c40-bbd98472ad4d None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:54.587150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1236b4-090f-4b6c-8c40-bbd98472ad4d None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:54.587552 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1236b4-090f-4b6c-8c40-bbd98472ad4d None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:54.589407 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 509/1026] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:54.595734 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260e802-63c8-4909-9f76-be79f8c38af0 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:54.595993 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260e802-63c8-4909-9f76-be79f8c38af0 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:54.596312 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260e802-63c8-4909-9f76-be79f8c38af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:54.615047 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-8260e802-63c8-4909-9f76-be79f8c38af0 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:54.616261 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-8260e802-63c8-4909-9f76-be79f8c38af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:54.627999 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939dcf8-9a2a-4820-af0b-f52020cf11b6 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:54.673296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974b8a2-0b12-4a3b-86df-88b7a26438f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:54.674061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974b8a2-0b12-4a3b-86df-88b7a26438f3 None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:54.674343 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974b8a2-0b12-4a3b-86df-88b7a26438f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:54.674615 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974b8a2-0b12-4a3b-86df-88b7a26438f3 None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:54.675279 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974b8a2-0b12-4a3b-86df-88b7a26438f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:54.676880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974b8a2-0b12-4a3b-86df-88b7a26438f3 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:54.703512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 510/1027] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:00:54.706237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 518/1028] 10.222.0.73 () {60 vars in 1132 bytes} [Mon Aug 29 20:00:54 2022] POST /identity/v3/users => generated 285 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:00:54.880721 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260e802-63c8-4909-9f76-be79f8c38af0 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:54.935796 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:54.986585 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974b8a2-0b12-4a3b-86df-88b7a26438f3 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:55.002534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260e802-63c8-4909-9f76-be79f8c38af0 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:55.048292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8597804-b8f3-44ff-ba90-042d0b9abca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:55.048923 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8597804-b8f3-44ff-ba90-042d0b9abca2 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:55.049206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8597804-b8f3-44ff-ba90-042d0b9abca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:55.049474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8597804-b8f3-44ff-ba90-042d0b9abca2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:55.060811 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8597804-b8f3-44ff-ba90-042d0b9abca2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:00:55.066596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-f8597804-b8f3-44ff-ba90-042d0b9abca2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:00:55.066596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:55.066596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:55.066596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:55.066596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:55.066596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:55.066596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:55.066596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:55.066596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:55.066596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:55.066596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:55.066596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:55.066596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:00:55.066596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:00:55.066596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:00:55.066596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:00:55.070287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:00:55.070287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:00:55.070287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:00:55.070287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:00:55.070287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:55.070287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:55.070287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:55.070287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:55.070287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:55.070287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:55.070287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:55.070287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:55.070287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:55.070287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:55.070287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:55.070287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:55.071206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:55.071206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:55.071206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:55.071206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:55.071206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:00:55.071206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:00:55.071206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:00:55.071206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:00:55.071206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:00:55.071206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:00:55.071206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:00:55.071206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:55.071206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 511/1029] 10.222.0.73 () {56 vars in 1120 bytes} [Mon Aug 29 20:00:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:55.075183 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8597804-b8f3-44ff-ba90-042d0b9abca2 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:55.105365 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:55.121284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974b8a2-0b12-4a3b-86df-88b7a26438f3 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:55.165783 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b55b87-3d65-4f1d-aa58-ea6b06a4a1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:55.166374 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b55b87-3d65-4f1d-aa58-ea6b06a4a1b0 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:55.166721 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b55b87-3d65-4f1d-aa58-ea6b06a4a1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:55.166999 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b55b87-3d65-4f1d-aa58-ea6b06a4a1b0 None admin] PATH_INFO: `/v3/services/image` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:55.167602 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b55b87-3d65-4f1d-aa58-ea6b06a4a1b0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:55.169013 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b55b87-3d65-4f1d-aa58-ea6b06a4a1b0 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:55.179323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-44b55b87-3d65-4f1d-aa58-ea6b06a4a1b0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:00:55.179323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:55.179323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:00:55.179323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:00:55.179323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:00:55.179323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:00:55.179323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:00:55.179323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:55.179323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:00:55.179323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:55.179323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:55.179323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:55.179323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:55.179323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:55.179323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:55.179323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:55.179323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:55.180443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:55.180443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:55.180443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:55.180443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:55.180443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:55.180443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:55.180443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:00:55.180443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:00:55.180443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:00:55.180443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:00:55.180443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:55.180443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:55.180443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:55.180443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:55.180443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:55.180443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:55.182894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:55.182894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:55.182894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:55.182894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:55.182894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:55.182894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:55.182894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:55.182894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:55.182894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:55.182894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:55.182894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:00:55.182894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:55.182894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:00:55.182894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:55.182894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 519/1030] 10.222.0.73 () {56 vars in 1105 bytes} [Mon Aug 29 20:00:54 2022] GET /identity/v3/services/image => generated 86 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:00:55.186751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b55b87-3d65-4f1d-aa58-ea6b06a4a1b0 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:55.234850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8597804-b8f3-44ff-ba90-042d0b9abca2 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:55.238906 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:55.282475 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4ccb5-9a60-47b9-8743-afa5d0f58c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:55.283126 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4ccb5-9a60-47b9-8743-afa5d0f58c5e None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:55.283406 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4ccb5-9a60-47b9-8743-afa5d0f58c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:55.283685 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4ccb5-9a60-47b9-8743-afa5d0f58c5e None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:55.284354 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a4ccb5-9a60-47b9-8743-afa5d0f58c5e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:55.284657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a4ccb5-9a60-47b9-8743-afa5d0f58c5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:55.286067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a4ccb5-9a60-47b9-8743-afa5d0f58c5e None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:55.297744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 512/1031] 10.222.0.73 () {56 vars in 1115 bytes} [Mon Aug 29 20:00:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:55.300853 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4ccb5-9a60-47b9-8743-afa5d0f58c5e None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:55.311293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b55b87-3d65-4f1d-aa58-ea6b06a4a1b0 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:55.353432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33ddf1-dd0f-4cd2-8ed5-ab9cd6600564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:55.354177 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33ddf1-dd0f-4cd2-8ed5-ab9cd6600564 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:55.354486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33ddf1-dd0f-4cd2-8ed5-ab9cd6600564 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:55.354914 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33ddf1-dd0f-4cd2-8ed5-ab9cd6600564 None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:55.356008 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33ddf1-dd0f-4cd2-8ed5-ab9cd6600564 None admin] RBAC: Adding query filter params (name=image) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:55.356554 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33ddf1-dd0f-4cd2-8ed5-ab9cd6600564 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:55.357909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33ddf1-dd0f-4cd2-8ed5-ab9cd6600564 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:55.369241 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 520/1032] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:55.370440 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33ddf1-dd0f-4cd2-8ed5-ab9cd6600564 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:55.407531 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:55.426114 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4ccb5-9a60-47b9-8743-afa5d0f58c5e None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:55.467397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813d1f0-519f-479f-b806-9d5a551f3843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:55.468848 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813d1f0-519f-479f-b806-9d5a551f3843 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:55.469168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813d1f0-519f-479f-b806-9d5a551f3843 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:55.469477 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813d1f0-519f-479f-b806-9d5a551f3843 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:55.483013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d813d1f0-519f-479f-b806-9d5a551f3843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:55.484975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d813d1f0-519f-479f-b806-9d5a551f3843 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:55.503375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-d813d1f0-519f-479f-b806-9d5a551f3843 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:55.503375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:55.503375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:55.503375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:55.503375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:55.503375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:55.503375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:55.503375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:55.503375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:55.503375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:55.503375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:55.503375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:55.503375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:55.503375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:55.503375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:55.503375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:55.504474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:55.504474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:55.504474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:55.504474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:55.504474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:55.504474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:55.504474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:55.504474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:55.504474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:55.504474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:55.504474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:55.504474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:55.504474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:55.504474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:55.504474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:55.504474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:55.504474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:55.505394 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:55.505394 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:55.505394 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:55.505394 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 513/1033] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:00:55.510443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813d1f0-519f-479f-b806-9d5a551f3843 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:55.516103 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33ddf1-dd0f-4cd2-8ed5-ab9cd6600564 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:55.532151 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:55.555951 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a0eb2-10aa-44a1-98df-dcd741cbe3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:55.556923 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a0eb2-10aa-44a1-98df-dcd741cbe3a8 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:55.557398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a0eb2-10aa-44a1-98df-dcd741cbe3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:55.557884 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a0eb2-10aa-44a1-98df-dcd741cbe3a8 None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:55.558707 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a0eb2-10aa-44a1-98df-dcd741cbe3a8 None admin] RBAC: Adding query filter params (type=image) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:55.559187 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a0eb2-10aa-44a1-98df-dcd741cbe3a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:55.560828 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a0eb2-10aa-44a1-98df-dcd741cbe3a8 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:55.566793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 521/1034] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:55 2022] GET /identity/v3/services?type=image => generated 350 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:55.572759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a0eb2-10aa-44a1-98df-dcd741cbe3a8 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:55.627878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813d1f0-519f-479f-b806-9d5a551f3843 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:55.669700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:55.678435 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d919a16-5690-43c4-82ba-f3492828ff05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:55.679312 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d919a16-5690-43c4-82ba-f3492828ff05 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:55.679758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d919a16-5690-43c4-82ba-f3492828ff05 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:55.680198 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d919a16-5690-43c4-82ba-f3492828ff05 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:55.681251 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d919a16-5690-43c4-82ba-f3492828ff05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:55.681771 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d919a16-5690-43c4-82ba-f3492828ff05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:55.683914 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d919a16-5690-43c4-82ba-f3492828ff05 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:55.704243 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a0eb2-10aa-44a1-98df-dcd741cbe3a8 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:55.709772 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 514/1035] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:55.713396 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d919a16-5690-43c4-82ba-f3492828ff05 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:55.747683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a71174-14cd-4c17-860c-3057ba96070c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:55.748398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a71174-14cd-4c17-860c-3057ba96070c None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:55.748686 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a71174-14cd-4c17-860c-3057ba96070c None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:55.749525 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a71174-14cd-4c17-860c-3057ba96070c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:55.750415 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a71174-14cd-4c17-860c-3057ba96070c None admin] RBAC: Adding query filter params (interface=public, service_id=a98f53e7410c473aa5a9fc687bf3467b, region_id=RegionOne) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:55.750845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a71174-14cd-4c17-860c-3057ba96070c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:55.752557 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a71174-14cd-4c17-860c-3057ba96070c None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:55.758492 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 522/1036] 10.222.0.73 () {56 vars in 1251 bytes} [Mon Aug 29 20:00:55 2022] GET /identity/v3/endpoints?service_id=a98f53e7410c473aa5a9fc687bf3467b&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:55.764000 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a71174-14cd-4c17-860c-3057ba96070c None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:55.799135 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:55.840764 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d919a16-5690-43c4-82ba-f3492828ff05 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:55.884786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda69b1-0a6f-47a2-bb0c-8c2096826eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:55.885435 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda69b1-0a6f-47a2-bb0c-8c2096826eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:55.885796 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda69b1-0a6f-47a2-bb0c-8c2096826eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:55.886073 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda69b1-0a6f-47a2-bb0c-8c2096826eb2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:55.892754 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a71174-14cd-4c17-860c-3057ba96070c None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:55.893990 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda69b1-0a6f-47a2-bb0c-8c2096826eb2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:55.895398 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda69b1-0a6f-47a2-bb0c-8c2096826eb2 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:55.907758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-3bda69b1-0a6f-47a2-bb0c-8c2096826eb2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:00:55.907758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:55.907758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:55.907758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:55.907758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:55.907758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:55.907758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:55.907758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:55.907758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:55.907758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:55.907758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:55.907758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:55.907758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:00:55.907758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:00:55.907758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:00:55.907758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:00:55.910056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:55.910056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:55.910056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:00:55.910056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:55.910056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:00:55.910056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:00:55.910056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:55.910056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:55.910056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:55.910056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:55.910056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:55.910056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:55.910056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:55.910056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:55.910056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:55.910056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:55.910056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:55.910056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:55.912359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:55.912359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:55.912359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:00:55.912359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:00:55.912359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:00:55.912359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:00:55.912359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:00:55.912359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:00:55.912359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:00:55.912359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:55.912359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 515/1037] 10.222.0.73 () {56 vars in 1093 bytes} [Mon Aug 29 20:00:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:00:55.919167 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda69b1-0a6f-47a2-bb0c-8c2096826eb2 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:55.948110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9b7c2-3ace-454a-81c9-861652ea077c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:55.948718 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9b7c2-3ace-454a-81c9-861652ea077c None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:55.948939 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9b7c2-3ace-454a-81c9-861652ea077c None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:55.949211 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9b7c2-3ace-454a-81c9-861652ea077c None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:55.949834 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca9b7c2-3ace-454a-81c9-861652ea077c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:55.951326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca9b7c2-3ace-454a-81c9-861652ea077c None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:55.966574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 523/1038] 10.222.0.73 () {56 vars in 1087 bytes} [Mon Aug 29 20:00:55 2022] GET /identity/v3/services => generated 1957 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:00:56.043657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda69b1-0a6f-47a2-bb0c-8c2096826eb2 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:56.082137 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5aef34-5929-4599-9a19-263f6ebbb669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:56.082763 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5aef34-5929-4599-9a19-263f6ebbb669 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:56.083044 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5aef34-5929-4599-9a19-263f6ebbb669 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:56.083317 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5aef34-5929-4599-9a19-263f6ebbb669 None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:56.084116 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5aef34-5929-4599-9a19-263f6ebbb669 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:56.084362 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5aef34-5929-4599-9a19-263f6ebbb669 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:56.085815 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5aef34-5929-4599-9a19-263f6ebbb669 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:56.099823 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:56.110166 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 516/1039] 10.222.0.73 () {56 vars in 1133 bytes} [Mon Aug 29 20:00:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:56.115946 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9b7c2-3ace-454a-81c9-861652ea077c None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:56.130432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:56.226844 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9b7c2-3ace-454a-81c9-861652ea077c None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:56.263703 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf132a9-1c13-445a-ba66-2e9400a36cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:56.264317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf132a9-1c13-445a-ba66-2e9400a36cde None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:56.264651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf132a9-1c13-445a-ba66-2e9400a36cde None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:56.264930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf132a9-1c13-445a-ba66-2e9400a36cde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:56.335354 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf132a9-1c13-445a-ba66-2e9400a36cde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:56.336808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf132a9-1c13-445a-ba66-2e9400a36cde None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:56.348076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:56.457533 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 524/1040] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:00:56 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:00:56.463103 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5aef34-5929-4599-9a19-263f6ebbb669 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:56.572173 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5aef34-5929-4599-9a19-263f6ebbb669 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:56.607181 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ccd6c-c7a9-4603-98ec-5f53bae84418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:56.607823 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ccd6c-c7a9-4603-98ec-5f53bae84418 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:56.608104 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ccd6c-c7a9-4603-98ec-5f53bae84418 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:56.609028 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ccd6c-c7a9-4603-98ec-5f53bae84418 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:56.619785 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ccd6c-c7a9-4603-98ec-5f53bae84418 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:56.621236 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ccd6c-c7a9-4603-98ec-5f53bae84418 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:56.636994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-c35ccd6c-c7a9-4603-98ec-5f53bae84418 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:56.636994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:56.636994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:56.636994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:56.636994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:56.636994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:56.636994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:56.636994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:56.636994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:56.636994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:56.636994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:56.636994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:56.636994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:00:56.636994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:00:56.636994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:00:56.636994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:00:56.638428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:56.638428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:56.638428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:56.638428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:56.638428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:56.638428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:56.638428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:56.638428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:56.638428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:56.638428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:56.638428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:56.638428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:56.638428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:56.638428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:56.638428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:56.638428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:56.638428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:00:56.640151 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:00:56.640151 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:00:56.640151 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:56.645751 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 517/1041] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:56.645936 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf132a9-1c13-445a-ba66-2e9400a36cde None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:56.756896 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf132a9-1c13-445a-ba66-2e9400a36cde None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:56.769152 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:56.794802 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ed0fe-1001-4035-9a73-774528cbb1dc None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:56.795089 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ed0fe-1001-4035-9a73-774528cbb1dc None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:56.795486 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ed0fe-1001-4035-9a73-774528cbb1dc None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:56.797066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 518/1042] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:00:56.798027 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc6f14-7c5c-4f06-99ab-a598a61fe8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:56.798701 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc6f14-7c5c-4f06-99ab-a598a61fe8ab None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:56.798980 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc6f14-7c5c-4f06-99ab-a598a61fe8ab None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:56.799280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc6f14-7c5c-4f06-99ab-a598a61fe8ab None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:56.799925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc6f14-7c5c-4f06-99ab-a598a61fe8ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:56.800226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc6f14-7c5c-4f06-99ab-a598a61fe8ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:56.801737 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc6f14-7c5c-4f06-99ab-a598a61fe8ab None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:56.802871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f5fb7-b6ea-42c7-ab8c-817aeda4af49 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:56.803571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f5fb7-b6ea-42c7-ab8c-817aeda4af49 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:56.804064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f5fb7-b6ea-42c7-ab8c-817aeda4af49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:56.814606 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 525/1043] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:56.820719 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc6f14-7c5c-4f06-99ab-a598a61fe8ab None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:56.826420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-256f5fb7-b6ea-42c7-ab8c-817aeda4af49 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:56.828118 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-256f5fb7-b6ea-42c7-ab8c-817aeda4af49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:56.890395 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:56.920460 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 519/1044] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:56.937019 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc6f14-7c5c-4f06-99ab-a598a61fe8ab None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:56.979435 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62064380-7a4f-479e-82f7-a30a14556e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:56.980067 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62064380-7a4f-479e-82f7-a30a14556e0d None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:56.980343 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62064380-7a4f-479e-82f7-a30a14556e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:56.980599 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62064380-7a4f-479e-82f7-a30a14556e0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:56.988012 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62064380-7a4f-479e-82f7-a30a14556e0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:56.989454 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62064380-7a4f-479e-82f7-a30a14556e0d None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:57.002609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-62064380-7a4f-479e-82f7-a30a14556e0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:57.002609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:57.002609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:57.002609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:57.002609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:57.002609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:57.002609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:57.002609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:57.002609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:57.002609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:57.002609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:57.002609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:57.002609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:00:57.002609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:00:57.002609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:00:57.003618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:00:57.003618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:57.003618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:57.003618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:57.003618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:57.003618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:57.003618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:57.003618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:57.003618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:57.003618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:57.003618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:57.003618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:57.003618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:57.003618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:57.003618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:57.003618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:57.003618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:57.004537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:00:57.004537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:00:57.004537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:00:57.004537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:00:57.004537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:00:57.004537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:00:57.004537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:00:57.004537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:00:57.005608 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 526/1045] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:00:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:00:57.009853 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f5fb7-b6ea-42c7-ab8c-817aeda4af49 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:57.132050 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f5fb7-b6ea-42c7-ab8c-817aeda4af49 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:57.167638 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ff98e-7c7e-4f94-8a80-40c2fad5a2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:57.168246 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ff98e-7c7e-4f94-8a80-40c2fad5a2ef None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:57.168521 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ff98e-7c7e-4f94-8a80-40c2fad5a2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:57.168787 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ff98e-7c7e-4f94-8a80-40c2fad5a2ef None admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:57.169474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ff98e-7c7e-4f94-8a80-40c2fad5a2ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:57.169810 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ff98e-7c7e-4f94-8a80-40c2fad5a2ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:57.171239 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ff98e-7c7e-4f94-8a80-40c2fad5a2ef None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:57.189668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 520/1046] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:00:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:57.191356 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62064380-7a4f-479e-82f7-a30a14556e0d None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:57.196111 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ff98e-7c7e-4f94-8a80-40c2fad5a2ef None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:57.227321 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:57.239374 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:57.310846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ff98e-7c7e-4f94-8a80-40c2fad5a2ef None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:57.325349 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62064380-7a4f-479e-82f7-a30a14556e0d None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:57.349884 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d744d71-4f8d-4e9e-ad10-59e4823794a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:57.350449 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d744d71-4f8d-4e9e-ad10-59e4823794a7 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:57.350736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d744d71-4f8d-4e9e-ad10-59e4823794a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:57.351020 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d744d71-4f8d-4e9e-ad10-59e4823794a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:57.364622 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d744d71-4f8d-4e9e-ad10-59e4823794a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:00:57.365636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9779d4-9acb-4469-a941-c7cbe49ed66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:57.366307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9779d4-9acb-4469-a941-c7cbe49ed66e None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:57.366588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9779d4-9acb-4469-a941-c7cbe49ed66e None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:57.366858 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9779d4-9acb-4469-a941-c7cbe49ed66e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:57.367588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9779d4-9acb-4469-a941-c7cbe49ed66e None admin] RBAC: Adding query filter params (role.id=64c60e8079b94f7b9cf2473b4b75f5a5, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=95cf77a92a174ff8b0503baf5c7e63be) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:57.367895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9779d4-9acb-4469-a941-c7cbe49ed66e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:57.369361 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9779d4-9acb-4469-a941-c7cbe49ed66e None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:57.369468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-8d744d71-4f8d-4e9e-ad10-59e4823794a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:00:57.369468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:57.369468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:57.369468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:57.369468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:57.369468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:57.369468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:57.369468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:57.369468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:57.369468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:57.369468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:57.369468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:57.369468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:00:57.369468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:00:57.369468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:00:57.369468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:00:57.370392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:00:57.370392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:00:57.370392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:00:57.370392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:00:57.370392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:57.370392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:57.370392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:57.370392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:57.370392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:57.370392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:57.370392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:57.370392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:57.370392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:57.370392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:57.370392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:57.370392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:57.375798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:57.375798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:57.375798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:57.375798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:57.375798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:00:57.375798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:00:57.375798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:00:57.375798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:00:57.375798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:00:57.375798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:00:57.375798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:00:57.375798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:57.375798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 521/1047] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 20:00:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:00:57.378005 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d744d71-4f8d-4e9e-ad10-59e4823794a7 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:57.382834 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 527/1048] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:00:57 2022] GET /identity/v3/role_assignments?user.id=95cf77a92a174ff8b0503baf5c7e63be&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=64c60e8079b94f7b9cf2473b4b75f5a5 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:57.523642 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:57.532331 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d744d71-4f8d-4e9e-ad10-59e4823794a7 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:57.591198 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f4756-772e-416d-ae04-29201a8c3eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:57.592194 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f4756-772e-416d-ae04-29201a8c3eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:57.592846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f4756-772e-416d-ae04-29201a8c3eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:57.593674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f4756-772e-416d-ae04-29201a8c3eb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:57.594717 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f4756-772e-416d-ae04-29201a8c3eb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:57.595250 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f4756-772e-416d-ae04-29201a8c3eb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:57.596825 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f4756-772e-416d-ae04-29201a8c3eb3 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:57.606960 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 522/1049] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 20:00:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:57.614961 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9779d4-9acb-4469-a941-c7cbe49ed66e None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:57.728793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:57.741507 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9779d4-9acb-4469-a941-c7cbe49ed66e None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:57.786909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c13d4f-4606-462a-9122-28b34beb139e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:57.787540 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c13d4f-4606-462a-9122-28b34beb139e None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:57.787814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c13d4f-4606-462a-9122-28b34beb139e None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:57.788186 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c13d4f-4606-462a-9122-28b34beb139e None admin] PATH_INFO: `/v3/users/dc071e73b29a4a86beb5e38a4a8ed754` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:57.803864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c13d4f-4606-462a-9122-28b34beb139e None admin] RBAC: Authorizing `identity:get_user(user_id=dc071e73b29a4a86beb5e38a4a8ed754)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:57.805314 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c13d4f-4606-462a-9122-28b34beb139e None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:57.807242 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 528/1050] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 20:00:57 2022] GET /identity/v3/users/dc071e73b29a4a86beb5e38a4a8ed754 => generated 285 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:57.813680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f4756-772e-416d-ae04-29201a8c3eb3 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:57.940126 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f4756-772e-416d-ae04-29201a8c3eb3 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:57.956611 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7cf17-dc8c-4e84-bae9-a3b1760ffd65 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:57.956972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7cf17-dc8c-4e84-bae9-a3b1760ffd65 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:57.957158 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7cf17-dc8c-4e84-bae9-a3b1760ffd65 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:57.958674 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 529/1051] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:57.963635 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013fe6b-4eab-4280-bc7a-3b86047f92c4 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:57.963916 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013fe6b-4eab-4280-bc7a-3b86047f92c4 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:57.964189 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013fe6b-4eab-4280-bc7a-3b86047f92c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:57.980467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4fd1bf-7447-4f13-a247-7480cbcf23e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:57.981085 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fd1bf-7447-4f13-a247-7480cbcf23e3 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:57.981367 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fd1bf-7447-4f13-a247-7480cbcf23e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:57.982906 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fd1bf-7447-4f13-a247-7480cbcf23e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:57.984973 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-5013fe6b-4eab-4280-bc7a-3b86047f92c4 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:57.986289 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-5013fe6b-4eab-4280-bc7a-3b86047f92c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:58.016376 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:58.058262 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 530/1052] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:00:58.064707 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4fd1bf-7447-4f13-a247-7480cbcf23e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:58.066514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4fd1bf-7447-4f13-a247-7480cbcf23e3 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:58.179169 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 523/1053] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:00:57 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:00:58.184800 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013fe6b-4eab-4280-bc7a-3b86047f92c4 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:58.273610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:58.294666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013fe6b-4eab-4280-bc7a-3b86047f92c4 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:58.331454 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd66006-556d-4a1d-abc4-1ed7dfc03a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:58.332083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd66006-556d-4a1d-abc4-1ed7dfc03a88 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:58.332359 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd66006-556d-4a1d-abc4-1ed7dfc03a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:58.332629 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd66006-556d-4a1d-abc4-1ed7dfc03a88 None admin] PATH_INFO: `/v3/projects/5b50615864e740e3a251e2af12ea3b06` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:58.333786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd66006-556d-4a1d-abc4-1ed7dfc03a88 None admin] RBAC: Authorizing `identity:get_project(project_id=5b50615864e740e3a251e2af12ea3b06)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:58.333910 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4fd1bf-7447-4f13-a247-7480cbcf23e3 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:58.335464 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd66006-556d-4a1d-abc4-1ed7dfc03a88 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:58.342283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 531/1054] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 20:00:58 2022] GET /identity/v3/projects/5b50615864e740e3a251e2af12ea3b06 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:58.343756 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd66006-556d-4a1d-abc4-1ed7dfc03a88 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:58.398710 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:58.443344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4fd1bf-7447-4f13-a247-7480cbcf23e3 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:58.473624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd66006-556d-4a1d-abc4-1ed7dfc03a88 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:58.479888 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb2ea6-5a9a-464b-a0eb-5736df57191d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:58.480521 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb2ea6-5a9a-464b-a0eb-5736df57191d None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:58.480836 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb2ea6-5a9a-464b-a0eb-5736df57191d None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:58.481110 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb2ea6-5a9a-464b-a0eb-5736df57191d None admin] PATH_INFO: `/v3/services/image` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:58.481690 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb2ea6-5a9a-464b-a0eb-5736df57191d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:58.483059 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb2ea6-5a9a-464b-a0eb-5736df57191d None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:58.493808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-31cb2ea6-5a9a-464b-a0eb-5736df57191d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:00:58.493808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:58.493808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:00:58.493808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:00:58.493808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:00:58.493808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:00:58.493808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:00:58.493808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:58.493808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:00:58.493808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:58.493808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:00:58.493808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:58.493808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:00:58.493808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:00:58.493808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:00:58.493808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:00:58.493808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:00:58.494908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:00:58.494908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:00:58.494908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:00:58.494908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:00:58.494908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:00:58.494908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:00:58.494908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:00:58.494908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:00:58.494908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:00:58.494908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:00:58.494908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:00:58.494908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:00:58.494908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:00:58.494908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:00:58.494908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:00:58.494908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:00:58.495750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:00:58.495750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:00:58.495750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:00:58.495750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:00:58.495750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:00:58.495750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:00:58.495750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:00:58.495750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:00:58.495750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:00:58.495750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:00:58.495750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:00:58.495750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:00:58.495750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:00:58.495750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:00:58.495750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 524/1055] 10.222.0.73 () {56 vars in 1105 bytes} [Mon Aug 29 20:00:58 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:00:58.501435 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb2ea6-5a9a-464b-a0eb-5736df57191d None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:58.513246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090cf64-6203-4990-be1a-c273b18aefd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:58.513950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090cf64-6203-4990-be1a-c273b18aefd8 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:58.514157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090cf64-6203-4990-be1a-c273b18aefd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:58.514424 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090cf64-6203-4990-be1a-c273b18aefd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:58.515436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0090cf64-6203-4990-be1a-c273b18aefd8 None admin] RBAC: Adding query filter params (role.id=296ee2c194824485af71618619f82e09, scope.project.id=5b50615864e740e3a251e2af12ea3b06, user.id=dc071e73b29a4a86beb5e38a4a8ed754) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:58.515936 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0090cf64-6203-4990-be1a-c273b18aefd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:58.517373 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0090cf64-6203-4990-be1a-c273b18aefd8 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:58.534258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 532/1056] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:00:58 2022] GET /identity/v3/role_assignments?user.id=dc071e73b29a4a86beb5e38a4a8ed754&scope.project.id=5b50615864e740e3a251e2af12ea3b06&role.id=296ee2c194824485af71618619f82e09 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:58.626984 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb2ea6-5a9a-464b-a0eb-5736df57191d None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:58.655517 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:58.668819 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3a0a3-8825-42b7-b4b4-641085fbe883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:58.669437 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3a0a3-8825-42b7-b4b4-641085fbe883 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:58.670141 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3a0a3-8825-42b7-b4b4-641085fbe883 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:58.670141 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3a0a3-8825-42b7-b4b4-641085fbe883 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:58.670641 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc3a0a3-8825-42b7-b4b4-641085fbe883 None admin] RBAC: Adding query filter params (name=image) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:58.670952 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc3a0a3-8825-42b7-b4b4-641085fbe883 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:58.673053 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc3a0a3-8825-42b7-b4b4-641085fbe883 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:58.678873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 525/1057] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:00:58.684402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090cf64-6203-4990-be1a-c273b18aefd8 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:58.701215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:58.795692 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090cf64-6203-4990-be1a-c273b18aefd8 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:58.832906 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eceb185-7d68-4f22-a10d-3874c918d5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:58.833530 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eceb185-7d68-4f22-a10d-3874c918d5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:58.833837 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eceb185-7d68-4f22-a10d-3874c918d5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:58.834110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eceb185-7d68-4f22-a10d-3874c918d5e9 None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:58.834758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eceb185-7d68-4f22-a10d-3874c918d5e9 None admin] RBAC: Adding query filter params (type=image) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:00:58.835052 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eceb185-7d68-4f22-a10d-3874c918d5e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:58.836445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eceb185-7d68-4f22-a10d-3874c918d5e9 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:58.849780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 533/1058] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:00:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:00:58.851392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3a0a3-8825-42b7-b4b4-641085fbe883 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:58.957648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3a0a3-8825-42b7-b4b4-641085fbe883 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:00:58.995236 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483e9d3-0884-47cc-963c-13cff137f878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:00:58.995855 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483e9d3-0884-47cc-963c-13cff137f878 None admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:58.996130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483e9d3-0884-47cc-963c-13cff137f878 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:58.996401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483e9d3-0884-47cc-963c-13cff137f878 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:58.997086 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d483e9d3-0884-47cc-963c-13cff137f878 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:00:58.998494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d483e9d3-0884-47cc-963c-13cff137f878 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:00:59.022256 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 526/1059] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:00:58 2022] POST /identity/v3/endpoints => generated 327 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:00:59.420738 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ea297-5471-4c42-b23c-bb6693d84f44 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:59.421255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ea297-5471-4c42-b23c-bb6693d84f44 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:59.421255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ea297-5471-4c42-b23c-bb6693d84f44 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:59.422608 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 534/1060] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:00:59.428098 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c33fe-154e-4db9-83ea-484bce5b0a89 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:00:59.428511 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c33fe-154e-4db9-83ea-484bce5b0a89 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:00:59.428808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c33fe-154e-4db9-83ea-484bce5b0a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:00:59.450000 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-602c33fe-154e-4db9-83ea-484bce5b0a89 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:00:59.451205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-602c33fe-154e-4db9-83ea-484bce5b0a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:00:59.500613 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:59.539034 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 527/1061] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:00:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:00:59.872014 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ea297-5471-4c42-b23c-bb6693d84f44 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:00:59.961123 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:00:59.987625 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ea297-5471-4c42-b23c-bb6693d84f44 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:00.035287 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc1554-9ca2-47aa-86eb-6e7a89406022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:00.036228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc1554-9ca2-47aa-86eb-6e7a89406022 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:00.036640 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc1554-9ca2-47aa-86eb-6e7a89406022 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:00.037117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc1554-9ca2-47aa-86eb-6e7a89406022 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:00.054260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc1554-9ca2-47aa-86eb-6e7a89406022 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:01:00.062320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-63cc1554-9ca2-47aa-86eb-6e7a89406022 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:01:00.062320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:01:00.062320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:01:00.062320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:01:00.062320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:01:00.062320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:01:00.062320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:01:00.062320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:01:00.062320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:01:00.062320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:01:00.062320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:01:00.062320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:01:00.062320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:01:00.062320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:01:00.062320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:01:00.062320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:01:00.063307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:01:00.063307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:01:00.063307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:01:00.063307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:01:00.063307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:01:00.063307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:01:00.063307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:01:00.063307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:01:00.063307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:01:00.063307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:01:00.063307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:01:00.063307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:01:00.063307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:01:00.063307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:01:00.063307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:01:00.063307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:01:00.064179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:01:00.064179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:01:00.064179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:01:00.064179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:01:00.064179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:01:00.064179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:01:00.064179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:01:00.064179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:01:00.064179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:01:00.064179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:01:00.064179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:01:00.064179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:01:00.065538 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 535/1062] 10.222.0.73 () {56 vars in 1120 bytes} [Mon Aug 29 20:00:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:01:00.072361 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c33fe-154e-4db9-83ea-484bce5b0a89 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:00.193491 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c33fe-154e-4db9-83ea-484bce5b0a89 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:00.229510 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae23fa-7dc5-4b6f-8c83-93bb03b082d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:00.230148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae23fa-7dc5-4b6f-8c83-93bb03b082d5 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:00.230425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae23fa-7dc5-4b6f-8c83-93bb03b082d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:00.230686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae23fa-7dc5-4b6f-8c83-93bb03b082d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:00.231346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae23fa-7dc5-4b6f-8c83-93bb03b082d5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:00.231647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae23fa-7dc5-4b6f-8c83-93bb03b082d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:00.232999 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae23fa-7dc5-4b6f-8c83-93bb03b082d5 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:00.252802 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 528/1063] 10.222.0.73 () {56 vars in 1115 bytes} [Mon Aug 29 20:01:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:00.258394 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc1554-9ca2-47aa-86eb-6e7a89406022 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:00.367131 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc1554-9ca2-47aa-86eb-6e7a89406022 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:00.403598 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30089c85-1763-427d-8390-5ade05bbe666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:00.404230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30089c85-1763-427d-8390-5ade05bbe666 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:00.404518 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30089c85-1763-427d-8390-5ade05bbe666 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:00.404794 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30089c85-1763-427d-8390-5ade05bbe666 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:00.408372 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:00.417421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30089c85-1763-427d-8390-5ade05bbe666 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:00.418915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30089c85-1763-427d-8390-5ade05bbe666 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:00.434797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-30089c85-1763-427d-8390-5ade05bbe666 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:01:00.434797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:01:00.434797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:01:00.434797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:01:00.434797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:01:00.434797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:01:00.434797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:01:00.434797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:01:00.434797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:01:00.434797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:01:00.434797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:01:00.434797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:01:00.434797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:01:00.434797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:01:00.434797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:01:00.434797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:01:00.436274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:01:00.436274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:01:00.436274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:01:00.436274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:01:00.436274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:01:00.436274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:01:00.436274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:01:00.436274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:01:00.436274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:01:00.436274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:01:00.436274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:01:00.436274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:01:00.436274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:01:00.436274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:01:00.436274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:01:00.436274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:01:00.436274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:01:00.437498 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:01:00.437498 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:01:00.437498 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:01:00.438347 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 536/1064] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:01:00.441970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae23fa-7dc5-4b6f-8c83-93bb03b082d5 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:00.477749 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:00.531325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881036ad-dba6-4398-83a1-457c3b9b423d None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:00.531710 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881036ad-dba6-4398-83a1-457c3b9b423d None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:00.531994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881036ad-dba6-4398-83a1-457c3b9b423d None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:00.533350 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 537/1065] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:01:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:01:00.538720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485a029-1b91-47c2-8460-c6edaaeb7797 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:00.539009 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485a029-1b91-47c2-8460-c6edaaeb7797 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:00.539286 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485a029-1b91-47c2-8460-c6edaaeb7797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:00.553699 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae23fa-7dc5-4b6f-8c83-93bb03b082d5 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:00.557576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-6485a029-1b91-47c2-8460-c6edaaeb7797 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:01:00.558843 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-6485a029-1b91-47c2-8460-c6edaaeb7797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:01:00.593008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c64fb-8003-413f-8ac8-6ac5563289d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:00.594451 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c64fb-8003-413f-8ac8-6ac5563289d4 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:00.594915 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c64fb-8003-413f-8ac8-6ac5563289d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:00.595370 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c64fb-8003-413f-8ac8-6ac5563289d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:00.596200 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c64fb-8003-413f-8ac8-6ac5563289d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:00.596678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c64fb-8003-413f-8ac8-6ac5563289d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:00.598331 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c64fb-8003-413f-8ac8-6ac5563289d4 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:00.612629 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 529/1066] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:01:00.618080 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c64fb-8003-413f-8ac8-6ac5563289d4 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:00.648579 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 538/1067] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:01:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:01:00.720204 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:00.730360 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c64fb-8003-413f-8ac8-6ac5563289d4 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:00.768483 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ec900-3d2d-4e60-a33b-9c5d75107ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:00.769123 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ec900-3d2d-4e60-a33b-9c5d75107ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:00.769406 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ec900-3d2d-4e60-a33b-9c5d75107ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:00.769676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ec900-3d2d-4e60-a33b-9c5d75107ba7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:00.777933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ec900-3d2d-4e60-a33b-9c5d75107ba7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:00.779386 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ec900-3d2d-4e60-a33b-9c5d75107ba7 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:00.791963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-fc4ec900-3d2d-4e60-a33b-9c5d75107ba7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:01:00.791963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:01:00.791963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:01:00.791963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:01:00.791963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:01:00.791963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:01:00.791963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:01:00.791963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:01:00.791963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:01:00.791963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:01:00.791963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:01:00.791963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:01:00.791963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:01:00.791963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:01:00.791963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:01:00.791963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:01:00.792934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:01:00.792934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:01:00.792934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:01:00.792934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:01:00.792934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:01:00.792934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:01:00.792934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:01:00.792934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:01:00.792934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:01:00.792934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:01:00.792934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:01:00.792934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:01:00.792934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:01:00.792934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:01:00.792934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:01:00.792934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:01:00.792934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:01:00.792934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:01:00.799727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:01:00.799727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:01:00.799727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:01:00.799727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:01:00.799727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:01:00.799727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:01:00.799727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:01:00.799727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:01:00.799727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:01:00.799727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:01:00.799727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 530/1068] 10.222.0.73 () {56 vars in 1093 bytes} [Mon Aug 29 20:01:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:01:00.800361 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485a029-1b91-47c2-8460-c6edaaeb7797 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:00.908660 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485a029-1b91-47c2-8460-c6edaaeb7797 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:00.925167 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ec900-3d2d-4e60-a33b-9c5d75107ba7 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:00.928661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:00.952774 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de01858-7b6c-4293-96df-74ed4d8eaf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:00.953394 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de01858-7b6c-4293-96df-74ed4d8eaf1d None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:00.953673 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de01858-7b6c-4293-96df-74ed4d8eaf1d None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:00.953975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de01858-7b6c-4293-96df-74ed4d8eaf1d None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:00.954709 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de01858-7b6c-4293-96df-74ed4d8eaf1d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:00.955013 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de01858-7b6c-4293-96df-74ed4d8eaf1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:00.956447 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de01858-7b6c-4293-96df-74ed4d8eaf1d None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:00.975934 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 539/1069] 10.222.0.73 () {56 vars in 1133 bytes} [Mon Aug 29 20:01:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:01:00.982131 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de01858-7b6c-4293-96df-74ed4d8eaf1d None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:01.048608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ec900-3d2d-4e60-a33b-9c5d75107ba7 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:01.087495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de01858-7b6c-4293-96df-74ed4d8eaf1d None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:01.087752 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9f7a7-502e-4872-a386-0b43dbb97d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:01.088227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9f7a7-502e-4872-a386-0b43dbb97d9c None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:01.088941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9f7a7-502e-4872-a386-0b43dbb97d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:01.088941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9f7a7-502e-4872-a386-0b43dbb97d9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:01.100891 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9f7a7-502e-4872-a386-0b43dbb97d9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:01:01.105755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-a0f9f7a7-502e-4872-a386-0b43dbb97d9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:01:01.105755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:01:01.105755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:01:01.105755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:01:01.105755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:01:01.105755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:01:01.105755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:01:01.105755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:01:01.105755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:01:01.105755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:01:01.105755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:01:01.105755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:01:01.105755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:01:01.105755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:01:01.105755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:01:01.105755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:01:01.106827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:01:01.106827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:01:01.106827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:01:01.106827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:01:01.106827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:01:01.106827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:01:01.106827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:01:01.106827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:01:01.106827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:01:01.106827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:01:01.106827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:01:01.106827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:01:01.106827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:01:01.106827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:01:01.106827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:01:01.106827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:01:01.107648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:01:01.107648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:01:01.107648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:01:01.107648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:01:01.107648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:01:01.107648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:01:01.107648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:01:01.107648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:01:01.107648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:01:01.107648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:01:01.107648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:01:01.107648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:01:01.107648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 531/1070] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 20:01:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:01:01.108560 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04690dd2-9265-4b25-b91c-c7c8ae7ad7ab None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:01.109652 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04690dd2-9265-4b25-b91c-c7c8ae7ad7ab None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:01.109861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04690dd2-9265-4b25-b91c-c7c8ae7ad7ab None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:01.111412 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 532/1071] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:01:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:01:01.113248 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04690dd2-9265-4b25-b91c-c7c8ae7ad7ab None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:01.129313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:01.129760 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fa84b-182e-4501-9d8b-85eb05ed2c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:01.130427 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fa84b-182e-4501-9d8b-85eb05ed2c59 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:01.130765 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fa84b-182e-4501-9d8b-85eb05ed2c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:01.131016 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fa84b-182e-4501-9d8b-85eb05ed2c59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:01.142148 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8fa84b-182e-4501-9d8b-85eb05ed2c59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:01.143707 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8fa84b-182e-4501-9d8b-85eb05ed2c59 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:01.159222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-0e8fa84b-182e-4501-9d8b-85eb05ed2c59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:01:01.159222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:01:01.159222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:01:01.159222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:01:01.159222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:01:01.159222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:01:01.159222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:01:01.159222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:01:01.159222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:01:01.159222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:01:01.159222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:01:01.159222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:01:01.159222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:01:01.159222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:01:01.159222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:01:01.159222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:01:01.160276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:01:01.160276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:01:01.160276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:01:01.160276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:01:01.160276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:01:01.160276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:01:01.160276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:01:01.160276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:01:01.160276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:01:01.160276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:01:01.160276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:01:01.160276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:01:01.160276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:01:01.160276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:01:01.160276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:01:01.160276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:01:01.160276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:01:01.161404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:01:01.161404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:01:01.161404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:01:01.161404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 540/1072] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:01:01.161909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8962890-69d1-4563-8d88-96adf1bf35b4 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:01.162193 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8962890-69d1-4563-8d88-96adf1bf35b4 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:01.162465 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8962890-69d1-4563-8d88-96adf1bf35b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:01.181017 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-c8962890-69d1-4563-8d88-96adf1bf35b4 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:01:01.182268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c8962890-69d1-4563-8d88-96adf1bf35b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:01:01.206619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:01.221686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04690dd2-9265-4b25-b91c-c7c8ae7ad7ab None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:01.252650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 541/1073] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:01:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:01:01.253383 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8962890-69d1-4563-8d88-96adf1bf35b4 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:01.261614 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa769b9-0f60-4a3d-969a-f42bb60276ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:01.262403 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa769b9-0f60-4a3d-969a-f42bb60276ec None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:01.262739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa769b9-0f60-4a3d-969a-f42bb60276ec None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:01.263011 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa769b9-0f60-4a3d-969a-f42bb60276ec None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:01.263826 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa769b9-0f60-4a3d-969a-f42bb60276ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:01.264132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa769b9-0f60-4a3d-969a-f42bb60276ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:01.266274 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa769b9-0f60-4a3d-969a-f42bb60276ec None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:01.278711 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 533/1074] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 20:01:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:01:01.284733 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa769b9-0f60-4a3d-969a-f42bb60276ec None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:01.368551 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:01.369329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8962890-69d1-4563-8d88-96adf1bf35b4 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:01.395776 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa769b9-0f60-4a3d-969a-f42bb60276ec None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:01.418581 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9a554-529c-498a-8289-d8f48d5cfad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:01.419184 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9a554-529c-498a-8289-d8f48d5cfad3 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:01.419462 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9a554-529c-498a-8289-d8f48d5cfad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:01.419734 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9a554-529c-498a-8289-d8f48d5cfad3 None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:01.420369 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa9a554-529c-498a-8289-d8f48d5cfad3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:01.420668 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa9a554-529c-498a-8289-d8f48d5cfad3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:01.422102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa9a554-529c-498a-8289-d8f48d5cfad3 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:01.434231 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 542/1075] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:01.435760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743075c-9a42-417e-98a0-bd3b074e5a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:01.436648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743075c-9a42-417e-98a0-bd3b074e5a56 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:01.437319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743075c-9a42-417e-98a0-bd3b074e5a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:01.437893 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743075c-9a42-417e-98a0-bd3b074e5a56 None admin] PATH_INFO: `/v3/users/dc071e73b29a4a86beb5e38a4a8ed754` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:01.439454 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743075c-9a42-417e-98a0-bd3b074e5a56 None admin] RBAC: Authorizing `identity:get_user(user_id=dc071e73b29a4a86beb5e38a4a8ed754)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:01.439634 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9a554-529c-498a-8289-d8f48d5cfad3 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:01.441540 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743075c-9a42-417e-98a0-bd3b074e5a56 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:01.443480 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 534/1076] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 20:01:01 2022] GET /identity/v3/users/dc071e73b29a4a86beb5e38a4a8ed754 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:01:01.449170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743075c-9a42-417e-98a0-bd3b074e5a56 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:01.466513 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:01.558720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9a554-529c-498a-8289-d8f48d5cfad3 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:01.563654 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743075c-9a42-417e-98a0-bd3b074e5a56 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:01.594321 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371f3f6-d0ef-4dde-99a1-5e682cb5eac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:01.594955 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371f3f6-d0ef-4dde-99a1-5e682cb5eac1 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:01.595233 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371f3f6-d0ef-4dde-99a1-5e682cb5eac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:01.595506 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371f3f6-d0ef-4dde-99a1-5e682cb5eac1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:01.601385 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fca284-a9ba-413f-b7be-72a4b0ea94c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:01.602058 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fca284-a9ba-413f-b7be-72a4b0ea94c3 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:01.602335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fca284-a9ba-413f-b7be-72a4b0ea94c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:01.602597 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8371f3f6-d0ef-4dde-99a1-5e682cb5eac1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:01.602710 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fca284-a9ba-413f-b7be-72a4b0ea94c3 None admin] PATH_INFO: `/v3/projects/5b50615864e740e3a251e2af12ea3b06` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:01.603732 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fca284-a9ba-413f-b7be-72a4b0ea94c3 None admin] RBAC: Authorizing `identity:get_project(project_id=5b50615864e740e3a251e2af12ea3b06)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:01.604050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8371f3f6-d0ef-4dde-99a1-5e682cb5eac1 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:01.605144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fca284-a9ba-413f-b7be-72a4b0ea94c3 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:01.606828 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 535/1077] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 20:01:01 2022] GET /identity/v3/projects/5b50615864e740e3a251e2af12ea3b06 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:01.610532 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fca284-a9ba-413f-b7be-72a4b0ea94c3 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:01.615921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-8371f3f6-d0ef-4dde-99a1-5e682cb5eac1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:01:01.615921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:01:01.615921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:01:01.615921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:01:01.615921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:01:01.615921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:01:01.615921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:01:01.615921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:01:01.615921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:01:01.615921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:01:01.615921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:01:01.615921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:01:01.615921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:01:01.615921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:01:01.615921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:01:01.616998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:01:01.616998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:01:01.616998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:01:01.616998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:01:01.616998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:01:01.616998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:01:01.616998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:01:01.616998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:01:01.616998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:01:01.616998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:01:01.616998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:01:01.616998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:01:01.616998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:01:01.616998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:01:01.616998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:01:01.616998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:01:01.616998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:01:01.617970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:01:01.617970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:01:01.617970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:01:01.617970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:01:01.617970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:01:01.617970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:01:01.617970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:01:01.617970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:01:01.617970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 543/1078] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:01:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:01:01.618455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371f3f6-d0ef-4dde-99a1-5e682cb5eac1 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:01.623872 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:01.704240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:01.719395 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fca284-a9ba-413f-b7be-72a4b0ea94c3 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:01.727472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371f3f6-d0ef-4dde-99a1-5e682cb5eac1 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:01.755843 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d038555-9309-4fe7-a88a-fe5eab472616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:01.756457 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d038555-9309-4fe7-a88a-fe5eab472616 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:01.756810 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d038555-9309-4fe7-a88a-fe5eab472616 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:01.757056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d038555-9309-4fe7-a88a-fe5eab472616 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:01.764674 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37349a93-990f-44dc-a6d0-aae816a4a1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:01.765998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37349a93-990f-44dc-a6d0-aae816a4a1a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:01.766480 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37349a93-990f-44dc-a6d0-aae816a4a1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:01.766763 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37349a93-990f-44dc-a6d0-aae816a4a1a9 None admin] PATH_INFO: `/v3/projects/5b50615864e740e3a251e2af12ea3b06/users/dc071e73b29a4a86beb5e38a4a8ed754/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:01.768821 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37349a93-990f-44dc-a6d0-aae816a4a1a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b50615864e740e3a251e2af12ea3b06, user_id=dc071e73b29a4a86beb5e38a4a8ed754, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:01.770312 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37349a93-990f-44dc-a6d0-aae816a4a1a9 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:01.784594 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 544/1079] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 20:01:01 2022] PUT /identity/v3/projects/5b50615864e740e3a251e2af12ea3b06/users/dc071e73b29a4a86beb5e38a4a8ed754/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:01:01.785412 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37349a93-990f-44dc-a6d0-aae816a4a1a9 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:01.833350 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d038555-9309-4fe7-a88a-fe5eab472616 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:01.834887 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d038555-9309-4fe7-a88a-fe5eab472616 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:01.850234 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:01.916769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37349a93-990f-44dc-a6d0-aae816a4a1a9 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:01.960950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0f337-f50e-4eea-818e-e67afb3d1acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:01.961572 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0f337-f50e-4eea-818e-e67afb3d1acc None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:01.961890 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0f337-f50e-4eea-818e-e67afb3d1acc None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:01.962156 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0f337-f50e-4eea-818e-e67afb3d1acc None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:01.962857 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0f337-f50e-4eea-818e-e67afb3d1acc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:01.963165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0f337-f50e-4eea-818e-e67afb3d1acc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:01.964609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0f337-f50e-4eea-818e-e67afb3d1acc None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:01.965268 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 536/1080] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:01:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:01:01.974800 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d038555-9309-4fe7-a88a-fe5eab472616 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:01.976682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:01.984166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 545/1081] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:01:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:01.990971 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0f337-f50e-4eea-818e-e67afb3d1acc None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:02.090935 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d038555-9309-4fe7-a88a-fe5eab472616 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:02.106479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:02.114838 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0f337-f50e-4eea-818e-e67afb3d1acc None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:02.133493 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f347ec3-2eac-4075-ac98-5c17e899decc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:02.134129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f347ec3-2eac-4075-ac98-5c17e899decc None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:02.134408 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f347ec3-2eac-4075-ac98-5c17e899decc None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:02.134683 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f347ec3-2eac-4075-ac98-5c17e899decc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:02.145946 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f347ec3-2eac-4075-ac98-5c17e899decc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:02.147402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f347ec3-2eac-4075-ac98-5c17e899decc None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:02.153309 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4447bf-66a9-4e3b-aa71-fe9bad700ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:02.154011 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4447bf-66a9-4e3b-aa71-fe9bad700ffb None admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:02.154290 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4447bf-66a9-4e3b-aa71-fe9bad700ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:02.154867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4447bf-66a9-4e3b-aa71-fe9bad700ffb None admin] PATH_INFO: `/v3/projects/2f5ec615d98548159487c97cead336a0/users/95cf77a92a174ff8b0503baf5c7e63be/roles/64c60e8079b94f7b9cf2473b4b75f5a5` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:02.157465 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4447bf-66a9-4e3b-aa71-fe9bad700ffb None admin] RBAC: Authorizing `identity:create_grant(project_id=2f5ec615d98548159487c97cead336a0, user_id=95cf77a92a174ff8b0503baf5c7e63be, role_id=64c60e8079b94f7b9cf2473b4b75f5a5)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:02.158933 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4447bf-66a9-4e3b-aa71-fe9bad700ffb None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:02.163310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-4f347ec3-2eac-4075-ac98-5c17e899decc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:01:02.163310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:01:02.163310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:01:02.163310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:01:02.163310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:01:02.163310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:01:02.163310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:01:02.163310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:01:02.163310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:01:02.163310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:01:02.163310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:01:02.163310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:01:02.163310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:01:02.163310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:01:02.163310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:01:02.163310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:01:02.164381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:01:02.164381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:01:02.164381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:01:02.164381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:01:02.164381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:01:02.164381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:01:02.164381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:01:02.164381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:01:02.164381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:01:02.164381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:01:02.164381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:01:02.164381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:01:02.164381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:01:02.164381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:01:02.164381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:01:02.164381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:01:02.164381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:01:02.165300 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:01:02.165300 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:01:02.165300 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:01:02.165300 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 537/1082] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:01:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:01:02.170653 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f347ec3-2eac-4075-ac98-5c17e899decc None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:02.181319 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 546/1083] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 20:01:01 2022] PUT /identity/v3/projects/2f5ec615d98548159487c97cead336a0/users/95cf77a92a174ff8b0503baf5c7e63be/roles/64c60e8079b94f7b9cf2473b4b75f5a5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:01:02.302401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f347ec3-2eac-4075-ac98-5c17e899decc None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:02.344362 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1224af0-363b-4601-b9fd-6886b81a9c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:02.345002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1224af0-363b-4601-b9fd-6886b81a9c30 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:02.345282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1224af0-363b-4601-b9fd-6886b81a9c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:02.345556 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1224af0-363b-4601-b9fd-6886b81a9c30 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:02.346285 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1224af0-363b-4601-b9fd-6886b81a9c30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:02.346590 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1224af0-363b-4601-b9fd-6886b81a9c30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:02.347967 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1224af0-363b-4601-b9fd-6886b81a9c30 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:02.364839 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 538/1084] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:01:03.737696 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646765d-2845-4464-9854-2502daa91b62 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:03.738258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646765d-2845-4464-9854-2502daa91b62 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:03.738388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646765d-2845-4464-9854-2502daa91b62 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:03.739725 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 547/1085] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:01:03.745191 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e2dce-588e-4cb9-9346-46bc25c6489f None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:03.745667 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e2dce-588e-4cb9-9346-46bc25c6489f None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:03.745808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e2dce-588e-4cb9-9346-46bc25c6489f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:03.766337 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-d87e2dce-588e-4cb9-9346-46bc25c6489f None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:01:03.767539 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d87e2dce-588e-4cb9-9346-46bc25c6489f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:01:03.788718 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:03.842198 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 539/1086] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:01:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:01:04.111320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646765d-2845-4464-9854-2502daa91b62 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:04.147653 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd90f0-2997-48c5-9e53-a0891cad6438 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:04.147948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd90f0-2997-48c5-9e53-a0891cad6438 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:04.148232 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd90f0-2997-48c5-9e53-a0891cad6438 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:04.149588 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 540/1087] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:01:04.154798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd28dd1-3167-4170-8d3d-35464f5fb616 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:04.155083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd28dd1-3167-4170-8d3d-35464f5fb616 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:04.155367 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd28dd1-3167-4170-8d3d-35464f5fb616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:04.173406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:04.173633 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-4fd28dd1-3167-4170-8d3d-35464f5fb616 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:01:04.175132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-4fd28dd1-3167-4170-8d3d-35464f5fb616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:01:04.218955 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646765d-2845-4464-9854-2502daa91b62 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:04.243414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 541/1088] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:01:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:01:04.255170 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6753f0-6ed8-464b-9c57-37b0d9cf0ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:04.255799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6753f0-6ed8-464b-9c57-37b0d9cf0ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:04.256073 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6753f0-6ed8-464b-9c57-37b0d9cf0ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:04.256341 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6753f0-6ed8-464b-9c57-37b0d9cf0ae8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:04.266798 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6753f0-6ed8-464b-9c57-37b0d9cf0ae8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:01:04.271522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-6f6753f0-6ed8-464b-9c57-37b0d9cf0ae8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:01:04.271522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:01:04.271522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:01:04.271522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:01:04.271522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:01:04.271522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:01:04.271522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:01:04.271522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:01:04.271522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:01:04.271522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:01:04.271522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:01:04.271522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:01:04.271522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:01:04.271522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:01:04.271522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:01:04.271522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:01:04.272639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:01:04.272639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:01:04.272639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:01:04.272639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:01:04.272639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:01:04.272639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:01:04.272639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:01:04.272639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:01:04.272639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:01:04.272639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:01:04.272639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:01:04.272639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:01:04.272639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:01:04.272639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:01:04.272639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:01:04.272639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:01:04.273505 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:01:04.273505 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:01:04.273505 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:01:04.273505 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:01:04.273505 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:01:04.273505 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:01:04.273505 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:01:04.273505 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:01:04.273505 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:01:04.273505 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:01:04.273505 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:01:04.273505 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:01:04.273505 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 548/1089] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 20:01:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:01:04.278536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd28dd1-3167-4170-8d3d-35464f5fb616 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:04.358380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:04.377942 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd28dd1-3167-4170-8d3d-35464f5fb616 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:04.414319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46ad4b-0beb-4c6b-af45-23116663e4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:04.414924 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46ad4b-0beb-4c6b-af45-23116663e4fc None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:04.415202 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46ad4b-0beb-4c6b-af45-23116663e4fc None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:04.415476 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46ad4b-0beb-4c6b-af45-23116663e4fc None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:04.416131 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46ad4b-0beb-4c6b-af45-23116663e4fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:04.416433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46ad4b-0beb-4c6b-af45-23116663e4fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:04.417819 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46ad4b-0beb-4c6b-af45-23116663e4fc None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:04.427151 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 542/1090] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 20:01:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:04.432879 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6753f0-6ed8-464b-9c57-37b0d9cf0ae8 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:04.512194 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46ad4b-0beb-4c6b-af45-23116663e4fc None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:04.537246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6753f0-6ed8-464b-9c57-37b0d9cf0ae8 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:04.556477 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:04.573540 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb190e-b6ab-441e-9ab0-5ef8a809c3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:04.574203 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb190e-b6ab-441e-9ab0-5ef8a809c3db None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:04.574722 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb190e-b6ab-441e-9ab0-5ef8a809c3db None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:04.574880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb190e-b6ab-441e-9ab0-5ef8a809c3db None admin] PATH_INFO: `/v3/users/dc071e73b29a4a86beb5e38a4a8ed754` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:04.576098 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffb190e-b6ab-441e-9ab0-5ef8a809c3db None admin] RBAC: Authorizing `identity:get_user(user_id=dc071e73b29a4a86beb5e38a4a8ed754)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:04.577696 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffb190e-b6ab-441e-9ab0-5ef8a809c3db None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:04.583388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 549/1091] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 20:01:04 2022] GET /identity/v3/users/dc071e73b29a4a86beb5e38a4a8ed754 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:04.585227 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb190e-b6ab-441e-9ab0-5ef8a809c3db None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:04.619879 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46ad4b-0beb-4c6b-af45-23116663e4fc None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:04.653391 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49c013-fb7f-4acb-b071-4ecd21cfe4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:04.654056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49c013-fb7f-4acb-b071-4ecd21cfe4a3 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:04.654339 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49c013-fb7f-4acb-b071-4ecd21cfe4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:04.654611 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49c013-fb7f-4acb-b071-4ecd21cfe4a3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:04.665839 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd49c013-fb7f-4acb-b071-4ecd21cfe4a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:01:04.670643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-dd49c013-fb7f-4acb-b071-4ecd21cfe4a3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:01:04.670643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:01:04.670643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:01:04.670643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:01:04.670643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:01:04.670643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:01:04.670643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:01:04.670643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:01:04.670643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:01:04.670643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:01:04.670643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:01:04.670643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:01:04.670643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:01:04.670643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:01:04.670643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:01:04.670643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:01:04.672208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:01:04.672208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:01:04.672208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:01:04.672208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:01:04.672208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:01:04.672208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:01:04.672208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:01:04.672208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:01:04.672208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:01:04.672208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:01:04.672208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:01:04.672208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:01:04.672208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:01:04.672208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:01:04.672208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:01:04.672208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:01:04.673066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:01:04.673066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:01:04.673066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:01:04.673066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:01:04.673066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:01:04.673066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:01:04.673066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:01:04.673066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:01:04.673066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:01:04.673066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:01:04.673066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:01:04.673066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:01:04.673066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 543/1092] 10.222.0.73 () {56 vars in 1120 bytes} [Mon Aug 29 20:01:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:01:04.678427 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49c013-fb7f-4acb-b071-4ecd21cfe4a3 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:04.681998 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:04.689764 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb190e-b6ab-441e-9ab0-5ef8a809c3db None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:04.724654 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827e01e-c036-41c9-bb39-3c64bf45cc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:04.725290 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827e01e-c036-41c9-bb39-3c64bf45cc02 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:04.725570 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827e01e-c036-41c9-bb39-3c64bf45cc02 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:04.725875 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827e01e-c036-41c9-bb39-3c64bf45cc02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:04.781874 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49c013-fb7f-4acb-b071-4ecd21cfe4a3 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:04.782449 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:04.797218 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827e01e-c036-41c9-bb39-3c64bf45cc02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:04.798793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827e01e-c036-41c9-bb39-3c64bf45cc02 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:04.817226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32b984-0266-42e0-8e7f-cdf986f48315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:04.817871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32b984-0266-42e0-8e7f-cdf986f48315 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:04.818151 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32b984-0266-42e0-8e7f-cdf986f48315 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:04.818423 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32b984-0266-42e0-8e7f-cdf986f48315 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:04.819083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32b984-0266-42e0-8e7f-cdf986f48315 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:04.819385 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32b984-0266-42e0-8e7f-cdf986f48315 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:04.820738 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32b984-0266-42e0-8e7f-cdf986f48315 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:04.829008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 544/1093] 10.222.0.73 () {56 vars in 1115 bytes} [Mon Aug 29 20:01:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:01:04.834670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32b984-0266-42e0-8e7f-cdf986f48315 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:04.900720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 550/1094] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:01:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 20:01:04.900912 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:04.907159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827e01e-c036-41c9-bb39-3c64bf45cc02 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:04.936491 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32b984-0266-42e0-8e7f-cdf986f48315 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:04.972402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609cd79-61cd-4b74-aec6-3b89b2433eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:04.973032 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609cd79-61cd-4b74-aec6-3b89b2433eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:04.973312 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609cd79-61cd-4b74-aec6-3b89b2433eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:04.973579 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609cd79-61cd-4b74-aec6-3b89b2433eb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:04.984239 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609cd79-61cd-4b74-aec6-3b89b2433eb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:04.985674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609cd79-61cd-4b74-aec6-3b89b2433eb3 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:04.993007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:05.000400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-9609cd79-61cd-4b74-aec6-3b89b2433eb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:01:05.000400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:01:05.000400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:01:05.000400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:01:05.000400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:01:05.000400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:01:05.000400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:01:05.000400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:01:05.000400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:01:05.000400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:01:05.000400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:01:05.000400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:01:05.000400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:01:05.000400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:01:05.000400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:01:05.000400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:01:05.001571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:01:05.001571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:01:05.001571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:01:05.001571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:01:05.001571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:01:05.001571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:01:05.001571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:01:05.001571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:01:05.001571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:01:05.001571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:01:05.001571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:01:05.001571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:01:05.001571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:01:05.001571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:01:05.001571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:01:05.001571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:01:05.001571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:01:05.002487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:01:05.002487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:01:05.002487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:01:05.002487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 545/1095] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:01:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:01:05.007654 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609cd79-61cd-4b74-aec6-3b89b2433eb3 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:05.011073 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827e01e-c036-41c9-bb39-3c64bf45cc02 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:05.046412 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71956398-534b-4a32-9601-7e50ccc964a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:05.047048 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71956398-534b-4a32-9601-7e50ccc964a8 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:05.047329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71956398-534b-4a32-9601-7e50ccc964a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:05.047605 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71956398-534b-4a32-9601-7e50ccc964a8 None admin] PATH_INFO: `/v3/projects/5b50615864e740e3a251e2af12ea3b06` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:05.048741 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71956398-534b-4a32-9601-7e50ccc964a8 None admin] RBAC: Authorizing `identity:get_project(project_id=5b50615864e740e3a251e2af12ea3b06)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:05.050319 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71956398-534b-4a32-9601-7e50ccc964a8 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:05.051966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 551/1096] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 20:01:04 2022] GET /identity/v3/projects/5b50615864e740e3a251e2af12ea3b06 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:01:05.058140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71956398-534b-4a32-9601-7e50ccc964a8 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:05.112997 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609cd79-61cd-4b74-aec6-3b89b2433eb3 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:05.130474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:05.149112 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea2b4d-0209-4a91-8a96-4680246093cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:05.149742 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea2b4d-0209-4a91-8a96-4680246093cb None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:05.150026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea2b4d-0209-4a91-8a96-4680246093cb None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:05.150297 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea2b4d-0209-4a91-8a96-4680246093cb None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:05.150989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea2b4d-0209-4a91-8a96-4680246093cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:05.151241 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea2b4d-0209-4a91-8a96-4680246093cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:05.152594 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea2b4d-0209-4a91-8a96-4680246093cb None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:05.164436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71956398-534b-4a32-9601-7e50ccc964a8 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:05.164808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 546/1097] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:01:05.170427 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea2b4d-0209-4a91-8a96-4680246093cb None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:05.198655 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e8360-47e5-43fc-bd84-b66d5d09006d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:05.199259 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e8360-47e5-43fc-bd84-b66d5d09006d None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:05.199544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e8360-47e5-43fc-bd84-b66d5d09006d None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:05.199817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e8360-47e5-43fc-bd84-b66d5d09006d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:05.200539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e8360-47e5-43fc-bd84-b66d5d09006d None admin] RBAC: Adding query filter params (role.id=296ee2c194824485af71618619f82e09, scope.project.id=5b50615864e740e3a251e2af12ea3b06, user.id=dc071e73b29a4a86beb5e38a4a8ed754) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:05.200844 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e8360-47e5-43fc-bd84-b66d5d09006d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:05.202342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e8360-47e5-43fc-bd84-b66d5d09006d None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:05.208761 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 552/1098] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:01:05 2022] GET /identity/v3/role_assignments?user.id=dc071e73b29a4a86beb5e38a4a8ed754&scope.project.id=5b50615864e740e3a251e2af12ea3b06&role.id=296ee2c194824485af71618619f82e09 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:01:05.275726 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea2b4d-0209-4a91-8a96-4680246093cb None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:05.312483 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5a3fd-df6c-4c85-8438-7bf3d2c028ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:05.313133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5a3fd-df6c-4c85-8438-7bf3d2c028ab None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:05.313417 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5a3fd-df6c-4c85-8438-7bf3d2c028ab None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:05.313692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5a3fd-df6c-4c85-8438-7bf3d2c028ab None admin] PATH_INFO: `/v3/users/nova` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:05.321821 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5a3fd-df6c-4c85-8438-7bf3d2c028ab None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:05.323283 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5a3fd-df6c-4c85-8438-7bf3d2c028ab None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:05.336538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-7aa5a3fd-df6c-4c85-8438-7bf3d2c028ab None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:01:05.336538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:01:05.336538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:01:05.336538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:01:05.336538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:01:05.336538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:01:05.336538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:01:05.336538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:01:05.336538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:01:05.336538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:01:05.336538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:01:05.336538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:01:05.336538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:01:05.336538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:01:05.336538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:01:05.336538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:01:05.337636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:01:05.337636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:01:05.337636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:01:05.337636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:01:05.337636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:01:05.337636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:01:05.337636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:01:05.337636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:01:05.337636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:01:05.337636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:01:05.337636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:01:05.337636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:01:05.337636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:01:05.337636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:01:05.337636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:01:05.337636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:01:05.337636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:01:05.337636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:01:05.338990 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:01:05.338990 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:01:05.338990 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:01:05.338990 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:01:05.338990 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:01:05.338990 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:01:05.338990 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:01:05.338990 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:01:05.338990 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:01:05.338990 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:01:05.338990 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 547/1099] 10.222.0.73 () {56 vars in 1093 bytes} [Mon Aug 29 20:01:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:01:05.343782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e8360-47e5-43fc-bd84-b66d5d09006d None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:05.366911 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:05.451797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e8360-47e5-43fc-bd84-b66d5d09006d None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:05.486677 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa616cb0-bd14-434e-b8fd-471cee4ecafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:05.487286 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa616cb0-bd14-434e-b8fd-471cee4ecafc None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:05.487564 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa616cb0-bd14-434e-b8fd-471cee4ecafc None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:05.487839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa616cb0-bd14-434e-b8fd-471cee4ecafc None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:05.488569 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa616cb0-bd14-434e-b8fd-471cee4ecafc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:05.488897 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa616cb0-bd14-434e-b8fd-471cee4ecafc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:05.490367 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa616cb0-bd14-434e-b8fd-471cee4ecafc None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:05.509738 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 553/1100] 10.222.0.73 () {56 vars in 1133 bytes} [Mon Aug 29 20:01:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:05.515456 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5a3fd-df6c-4c85-8438-7bf3d2c028ab None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:05.558149 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:05.629331 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5a3fd-df6c-4c85-8438-7bf3d2c028ab None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:05.663920 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d2e17-3148-42be-8f25-b11e719a3034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:05.664528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d2e17-3148-42be-8f25-b11e719a3034 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:05.664803 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d2e17-3148-42be-8f25-b11e719a3034 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:05.665078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d2e17-3148-42be-8f25-b11e719a3034 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:05.737224 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415d2e17-3148-42be-8f25-b11e719a3034 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:05.738616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415d2e17-3148-42be-8f25-b11e719a3034 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:05.778971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:05.843274 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 548/1101] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:01:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:01:05.849347 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa616cb0-bd14-434e-b8fd-471cee4ecafc None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:05.930927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:05.955176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa616cb0-bd14-434e-b8fd-471cee4ecafc None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:05.991039 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd50f0d-f74d-46dd-9a90-3af17bf3edce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:05.991571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd50f0d-f74d-46dd-9a90-3af17bf3edce None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:05.991847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd50f0d-f74d-46dd-9a90-3af17bf3edce None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:05.992124 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd50f0d-f74d-46dd-9a90-3af17bf3edce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:06.003750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd50f0d-f74d-46dd-9a90-3af17bf3edce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:06.005196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd50f0d-f74d-46dd-9a90-3af17bf3edce None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:06.020213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-1bd50f0d-f74d-46dd-9a90-3af17bf3edce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:01:06.020213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:01:06.020213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:01:06.020213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:01:06.020213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:01:06.020213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:01:06.020213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:01:06.020213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:01:06.020213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:01:06.020213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:01:06.020213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:01:06.020213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:01:06.020213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:01:06.020213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:01:06.020213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:01:06.020213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:01:06.021198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:01:06.021198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:01:06.021198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:01:06.021198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:01:06.021198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:01:06.021198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:01:06.021198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:01:06.021198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:01:06.021198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:01:06.021198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:01:06.021198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:01:06.021198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:01:06.021198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:01:06.021198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:01:06.021198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:01:06.021198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:01:06.021198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:01:06.022332 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:01:06.022332 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:01:06.022332 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:01:06.022332 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 554/1102] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:01:06.027218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d2e17-3148-42be-8f25-b11e719a3034 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:06.137234 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d2e17-3148-42be-8f25-b11e719a3034 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:06.173872 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d2b197-f965-4615-98ab-7fa4a383f410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:06.174482 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2b197-f965-4615-98ab-7fa4a383f410 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:06.174765 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2b197-f965-4615-98ab-7fa4a383f410 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:06.175044 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2b197-f965-4615-98ab-7fa4a383f410 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:06.175688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d2b197-f965-4615-98ab-7fa4a383f410 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:06.175993 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d2b197-f965-4615-98ab-7fa4a383f410 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:06.177354 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d2b197-f965-4615-98ab-7fa4a383f410 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:06.183971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:06.190009 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 549/1103] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:06.195857 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd50f0d-f74d-46dd-9a90-3af17bf3edce None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:06.298427 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd50f0d-f74d-46dd-9a90-3af17bf3edce None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:06.324278 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:06.333207 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039ba29-05bf-4a4d-9b03-657c93b3e7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:06.333886 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039ba29-05bf-4a4d-9b03-657c93b3e7f5 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:06.334166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039ba29-05bf-4a4d-9b03-657c93b3e7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:06.334436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039ba29-05bf-4a4d-9b03-657c93b3e7f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:06.342171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6039ba29-05bf-4a4d-9b03-657c93b3e7f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:06.344546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6039ba29-05bf-4a4d-9b03-657c93b3e7f5 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:06.356296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-6039ba29-05bf-4a4d-9b03-657c93b3e7f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:01:06.356296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:01:06.356296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:01:06.356296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:01:06.356296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:01:06.356296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:01:06.356296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:01:06.356296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:01:06.356296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:01:06.356296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:01:06.356296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:01:06.356296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:01:06.356296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:01:06.356296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:01:06.356296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:01:06.357392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:01:06.357392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:01:06.357392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:01:06.357392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:01:06.357392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:01:06.357392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:01:06.357392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:01:06.357392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:01:06.357392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:01:06.357392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:01:06.357392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:01:06.357392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:01:06.357392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:01:06.357392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:01:06.357392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:01:06.357392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:01:06.357392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:01:06.358732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:01:06.358732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:01:06.358732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:01:06.358732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:01:06.358732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:01:06.358732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:01:06.358732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:01:06.358732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:01:06.358732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 555/1104] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:01:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:01:06.364291 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d2b197-f965-4615-98ab-7fa4a383f410 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:06.467309 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d2b197-f965-4615-98ab-7fa4a383f410 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:06.500624 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1698e-3c9b-40f5-9bc6-3cb5a53e7aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:06.501228 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1698e-3c9b-40f5-9bc6-3cb5a53e7aaa None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:06.501503 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1698e-3c9b-40f5-9bc6-3cb5a53e7aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:06.501803 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1698e-3c9b-40f5-9bc6-3cb5a53e7aaa None admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:06.502488 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1698e-3c9b-40f5-9bc6-3cb5a53e7aaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:06.503059 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1698e-3c9b-40f5-9bc6-3cb5a53e7aaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:06.504482 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1698e-3c9b-40f5-9bc6-3cb5a53e7aaa None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:06.517162 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 550/1105] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:01:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:06.523159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039ba29-05bf-4a4d-9b03-657c93b3e7f5 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:06.623340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039ba29-05bf-4a4d-9b03-657c93b3e7f5 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:06.658756 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec6473-4a3a-43d7-bfac-1a18d59734b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:06.659358 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec6473-4a3a-43d7-bfac-1a18d59734b2 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:06.659638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec6473-4a3a-43d7-bfac-1a18d59734b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:06.659907 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec6473-4a3a-43d7-bfac-1a18d59734b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:06.660632 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec6473-4a3a-43d7-bfac-1a18d59734b2 None admin] RBAC: Adding query filter params (role.id=64c60e8079b94f7b9cf2473b4b75f5a5, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=95cf77a92a174ff8b0503baf5c7e63be) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:06.660933 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec6473-4a3a-43d7-bfac-1a18d59734b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:06.662401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec6473-4a3a-43d7-bfac-1a18d59734b2 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:06.668417 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 556/1106] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:01:06 2022] GET /identity/v3/role_assignments?user.id=95cf77a92a174ff8b0503baf5c7e63be&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=64c60e8079b94f7b9cf2473b4b75f5a5 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:07.189307 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159a487-94c1-4831-8307-794eb34865a1 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:07.189890 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159a487-94c1-4831-8307-794eb34865a1 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:07.189984 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159a487-94c1-4831-8307-794eb34865a1 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:07.191394 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 551/1107] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:01:07.197472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1629b01-6151-463a-bd0d-f2cfe10fd97d None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:07.197997 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1629b01-6151-463a-bd0d-f2cfe10fd97d None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:07.198086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1629b01-6151-463a-bd0d-f2cfe10fd97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:07.217456 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-c1629b01-6151-463a-bd0d-f2cfe10fd97d None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:01:07.218797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c1629b01-6151-463a-bd0d-f2cfe10fd97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:01:07.275465 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:07.299413 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 557/1108] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:01:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:01:07.574811 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159a487-94c1-4831-8307-794eb34865a1 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:07.636609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:07.680490 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159a487-94c1-4831-8307-794eb34865a1 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:07.714204 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a1b90-2ce1-4011-a0cf-28f751daadd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:07.714827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a1b90-2ce1-4011-a0cf-28f751daadd1 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:07.715103 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a1b90-2ce1-4011-a0cf-28f751daadd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:07.715370 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a1b90-2ce1-4011-a0cf-28f751daadd1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:07.716471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a1b90-2ce1-4011-a0cf-28f751daadd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:07.717920 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a1b90-2ce1-4011-a0cf-28f751daadd1 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:07.720387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 552/1109] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:01:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:07.726186 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1629b01-6151-463a-bd0d-f2cfe10fd97d None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:07.826224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1629b01-6151-463a-bd0d-f2cfe10fd97d None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:07.862200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263ce05-e319-4478-acef-f65e47a66929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:07.862807 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263ce05-e319-4478-acef-f65e47a66929 None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:07.863081 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263ce05-e319-4478-acef-f65e47a66929 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:07.863348 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263ce05-e319-4478-acef-f65e47a66929 None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:07.863994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2263ce05-e319-4478-acef-f65e47a66929 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:07.865429 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2263ce05-e319-4478-acef-f65e47a66929 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:07.888610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 558/1110] 10.222.0.73 () {60 vars in 1132 bytes} [Mon Aug 29 20:01:07 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:01:09.865056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eda42a-5321-4e0b-98ef-95f597ecb8d7 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:09.865592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eda42a-5321-4e0b-98ef-95f597ecb8d7 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:09.865592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eda42a-5321-4e0b-98ef-95f597ecb8d7 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:09.867020 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 553/1111] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:01:09.872354 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ff350-e960-4b03-b376-219338bef0aa None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:09.872775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ff350-e960-4b03-b376-219338bef0aa None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:09.872882 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ff350-e960-4b03-b376-219338bef0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:09.892283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-a80ff350-e960-4b03-b376-219338bef0aa None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:01:09.893480 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a80ff350-e960-4b03-b376-219338bef0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:01:09.923957 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:09.962772 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 559/1112] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:01:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:01:10.230955 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eda42a-5321-4e0b-98ef-95f597ecb8d7 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:10.336377 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eda42a-5321-4e0b-98ef-95f597ecb8d7 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:10.362427 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:10.371625 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82362b3b-2584-4dc3-bf54-93cc0609ba1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:10.372257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82362b3b-2584-4dc3-bf54-93cc0609ba1e None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:10.372534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82362b3b-2584-4dc3-bf54-93cc0609ba1e None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:10.372810 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82362b3b-2584-4dc3-bf54-93cc0609ba1e None admin] PATH_INFO: `/v3/roles/b9184599d95b4950beca34304cdd2eb8` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:10.374247 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82362b3b-2584-4dc3-bf54-93cc0609ba1e None admin] RBAC: Authorizing `identity:get_role(role_id=b9184599d95b4950beca34304cdd2eb8)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:10.376394 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82362b3b-2584-4dc3-bf54-93cc0609ba1e None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:10.377759 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 554/1113] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 20:01:10 2022] GET /identity/v3/roles/b9184599d95b4950beca34304cdd2eb8 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:10.383838 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ff350-e960-4b03-b376-219338bef0aa None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:10.482195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ff350-e960-4b03-b376-219338bef0aa None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:10.521375 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80494bd-684e-4d31-8472-1bf6fd270cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:10.522024 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80494bd-684e-4d31-8472-1bf6fd270cdf None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:10.522310 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80494bd-684e-4d31-8472-1bf6fd270cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:10.522582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80494bd-684e-4d31-8472-1bf6fd270cdf None admin] PATH_INFO: `/v3/users/dddf1df5ba7d4b1c8c097547af0c29da` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:10.535792 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80494bd-684e-4d31-8472-1bf6fd270cdf None admin] RBAC: Authorizing `identity:get_user(user_id=dddf1df5ba7d4b1c8c097547af0c29da)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:10.537228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80494bd-684e-4d31-8472-1bf6fd270cdf None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:10.538871 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 560/1114] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 20:01:10 2022] GET /identity/v3/users/dddf1df5ba7d4b1c8c097547af0c29da => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:10.544685 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82362b3b-2584-4dc3-bf54-93cc0609ba1e None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:10.642941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82362b3b-2584-4dc3-bf54-93cc0609ba1e None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:10.677060 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7044a-88ab-485d-8b4e-119ac9ddb31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:10.677657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7044a-88ab-485d-8b4e-119ac9ddb31b None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:10.677965 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7044a-88ab-485d-8b4e-119ac9ddb31b None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:10.678236 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7044a-88ab-485d-8b4e-119ac9ddb31b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:10.736547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:10.744039 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7044a-88ab-485d-8b4e-119ac9ddb31b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:10.745467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7044a-88ab-485d-8b4e-119ac9ddb31b None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:10.845035 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 555/1115] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:01:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:01:10.851195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80494bd-684e-4d31-8472-1bf6fd270cdf None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:10.880517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:10.952473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80494bd-684e-4d31-8472-1bf6fd270cdf None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:10.986802 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c68b82-ba9b-4bea-8009-89b3d42f133a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:10.987352 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c68b82-ba9b-4bea-8009-89b3d42f133a None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:10.987629 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c68b82-ba9b-4bea-8009-89b3d42f133a None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:10.987903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c68b82-ba9b-4bea-8009-89b3d42f133a None admin] PATH_INFO: `/v3/projects/5b50615864e740e3a251e2af12ea3b06` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:10.989013 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c68b82-ba9b-4bea-8009-89b3d42f133a None admin] RBAC: Authorizing `identity:get_project(project_id=5b50615864e740e3a251e2af12ea3b06)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:10.990479 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c68b82-ba9b-4bea-8009-89b3d42f133a None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:10.992105 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 561/1116] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 20:01:10 2022] GET /identity/v3/projects/5b50615864e740e3a251e2af12ea3b06 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:01:10.998315 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7044a-88ab-485d-8b4e-119ac9ddb31b None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:11.090841 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:11.098443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7044a-88ab-485d-8b4e-119ac9ddb31b None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:11.132363 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9535778-fc47-4772-8a46-ff164aeb1f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:11.132948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9535778-fc47-4772-8a46-ff164aeb1f4f None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:11.133220 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9535778-fc47-4772-8a46-ff164aeb1f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:11.133484 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9535778-fc47-4772-8a46-ff164aeb1f4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:11.134225 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9535778-fc47-4772-8a46-ff164aeb1f4f None admin] RBAC: Adding query filter params (role.id=b9184599d95b4950beca34304cdd2eb8, scope.project.id=5b50615864e740e3a251e2af12ea3b06, user.id=dddf1df5ba7d4b1c8c097547af0c29da) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:11.134528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9535778-fc47-4772-8a46-ff164aeb1f4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:11.135937 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9535778-fc47-4772-8a46-ff164aeb1f4f None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:11.141477 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 556/1117] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:01:10 2022] GET /identity/v3/role_assignments?user.id=dddf1df5ba7d4b1c8c097547af0c29da&scope.project.id=5b50615864e740e3a251e2af12ea3b06&role.id=b9184599d95b4950beca34304cdd2eb8 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:13.066700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d58e06-ba1a-4c2b-b5ea-8e19a3187f3f None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:13.067274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d58e06-ba1a-4c2b-b5ea-8e19a3187f3f None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:13.067274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d58e06-ba1a-4c2b-b5ea-8e19a3187f3f None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:13.068570 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 562/1118] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:01:13.073889 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda38ae4-eefc-44e0-93c2-daba7992b6bf None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:13.074334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda38ae4-eefc-44e0-93c2-daba7992b6bf None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:13.074427 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda38ae4-eefc-44e0-93c2-daba7992b6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:13.093555 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-fda38ae4-eefc-44e0-93c2-daba7992b6bf None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:01:13.094819 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-fda38ae4-eefc-44e0-93c2-daba7992b6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:01:13.161584 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 557/1119] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:01:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:01:13.432110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d58e06-ba1a-4c2b-b5ea-8e19a3187f3f None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:13.529388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:13.538135 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d58e06-ba1a-4c2b-b5ea-8e19a3187f3f None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:13.582542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd5b045-8903-4909-95aa-438564e23504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:13.583165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5b045-8903-4909-95aa-438564e23504 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:13.583438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5b045-8903-4909-95aa-438564e23504 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:13.583706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5b045-8903-4909-95aa-438564e23504 None admin] PATH_INFO: `/v3/roles/b9184599d95b4950beca34304cdd2eb8` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:13.585098 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd5b045-8903-4909-95aa-438564e23504 None admin] RBAC: Authorizing `identity:get_role(role_id=b9184599d95b4950beca34304cdd2eb8)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:13.586500 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd5b045-8903-4909-95aa-438564e23504 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:13.587827 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 563/1120] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 20:01:13 2022] GET /identity/v3/roles/b9184599d95b4950beca34304cdd2eb8 => generated 223 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:13.594366 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda38ae4-eefc-44e0-93c2-daba7992b6bf None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:13.684713 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:13.698350 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda38ae4-eefc-44e0-93c2-daba7992b6bf None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:13.732915 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062a4f7-36df-4493-8ba7-a1c50805fbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:13.733548 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062a4f7-36df-4493-8ba7-a1c50805fbd5 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:13.733863 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062a4f7-36df-4493-8ba7-a1c50805fbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:13.734716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062a4f7-36df-4493-8ba7-a1c50805fbd5 None admin] PATH_INFO: `/v3/users/dddf1df5ba7d4b1c8c097547af0c29da` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:13.735862 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4062a4f7-36df-4493-8ba7-a1c50805fbd5 None admin] RBAC: Authorizing `identity:get_user(user_id=dddf1df5ba7d4b1c8c097547af0c29da)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:13.737359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4062a4f7-36df-4493-8ba7-a1c50805fbd5 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:13.738993 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 558/1121] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 20:01:13 2022] GET /identity/v3/users/dddf1df5ba7d4b1c8c097547af0c29da => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:13.744761 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd5b045-8903-4909-95aa-438564e23504 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:13.850512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd5b045-8903-4909-95aa-438564e23504 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:13.885168 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086939a0-a421-4519-9ebd-878de4335841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:13.885833 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086939a0-a421-4519-9ebd-878de4335841 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:13.886117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086939a0-a421-4519-9ebd-878de4335841 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:13.886450 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086939a0-a421-4519-9ebd-878de4335841 None admin] PATH_INFO: `/v3/projects/5b50615864e740e3a251e2af12ea3b06` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:13.887596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086939a0-a421-4519-9ebd-878de4335841 None admin] RBAC: Authorizing `identity:get_project(project_id=5b50615864e740e3a251e2af12ea3b06)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:13.889027 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086939a0-a421-4519-9ebd-878de4335841 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:13.891302 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 564/1122] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 20:01:13 2022] GET /identity/v3/projects/5b50615864e740e3a251e2af12ea3b06 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:13.897527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062a4f7-36df-4493-8ba7-a1c50805fbd5 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:13.998063 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062a4f7-36df-4493-8ba7-a1c50805fbd5 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:14.033848 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e1ea7-f15c-4631-bc67-89a6131e342c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:14.034504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e1ea7-f15c-4631-bc67-89a6131e342c None admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:14.034784 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e1ea7-f15c-4631-bc67-89a6131e342c None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:14.035053 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e1ea7-f15c-4631-bc67-89a6131e342c None admin] PATH_INFO: `/v3/projects/5b50615864e740e3a251e2af12ea3b06/users/dddf1df5ba7d4b1c8c097547af0c29da/roles/b9184599d95b4950beca34304cdd2eb8` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:14.037363 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160e1ea7-f15c-4631-bc67-89a6131e342c None admin] RBAC: Authorizing `identity:create_grant(project_id=5b50615864e740e3a251e2af12ea3b06, user_id=dddf1df5ba7d4b1c8c097547af0c29da, role_id=b9184599d95b4950beca34304cdd2eb8)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:14.038829 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160e1ea7-f15c-4631-bc67-89a6131e342c None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:14.053542 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 559/1123] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 20:01:13 2022] PUT /identity/v3/projects/5b50615864e740e3a251e2af12ea3b06/users/dddf1df5ba7d4b1c8c097547af0c29da/roles/b9184599d95b4950beca34304cdd2eb8 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:01:15.949066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00be54-98c2-4087-83b2-7584712b1dd3 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:15.949635 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00be54-98c2-4087-83b2-7584712b1dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:15.949635 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00be54-98c2-4087-83b2-7584712b1dd3 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:15.950970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 565/1124] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:01:15.957361 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcb8db-8243-4346-978e-c899605f5637 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:15.957828 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcb8db-8243-4346-978e-c899605f5637 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:15.957931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcb8db-8243-4346-978e-c899605f5637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:15.975852 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-23bcb8db-8243-4346-978e-c899605f5637 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:01:15.977027 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-23bcb8db-8243-4346-978e-c899605f5637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:01:15.987682 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:16.050570 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 560/1125] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:01:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:01:16.322544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00be54-98c2-4087-83b2-7584712b1dd3 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:16.353849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:16.426028 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00be54-98c2-4087-83b2-7584712b1dd3 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:16.460057 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdd5a7-68b1-4f57-a495-c2d0d39da954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:16.460675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdd5a7-68b1-4f57-a495-c2d0d39da954 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:16.460956 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdd5a7-68b1-4f57-a495-c2d0d39da954 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:16.461231 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdd5a7-68b1-4f57-a495-c2d0d39da954 None admin] PATH_INFO: `/v3/roles/b9184599d95b4950beca34304cdd2eb8` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:16.462635 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcdd5a7-68b1-4f57-a495-c2d0d39da954 None admin] RBAC: Authorizing `identity:get_role(role_id=b9184599d95b4950beca34304cdd2eb8)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:16.464002 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcdd5a7-68b1-4f57-a495-c2d0d39da954 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:16.465319 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 566/1126] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 20:01:16 2022] GET /identity/v3/roles/b9184599d95b4950beca34304cdd2eb8 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:16.471185 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcb8db-8243-4346-978e-c899605f5637 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:16.567648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcb8db-8243-4346-978e-c899605f5637 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:16.600448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f0e37-3e21-40c8-b0f6-5eefb3f152f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:16.601066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f0e37-3e21-40c8-b0f6-5eefb3f152f3 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:16.601341 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f0e37-3e21-40c8-b0f6-5eefb3f152f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:16.601607 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f0e37-3e21-40c8-b0f6-5eefb3f152f3 None admin] PATH_INFO: `/v3/users/dddf1df5ba7d4b1c8c097547af0c29da` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:16.602763 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f0e37-3e21-40c8-b0f6-5eefb3f152f3 None admin] RBAC: Authorizing `identity:get_user(user_id=dddf1df5ba7d4b1c8c097547af0c29da)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:16.604176 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f0e37-3e21-40c8-b0f6-5eefb3f152f3 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:16.605756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 561/1127] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 20:01:16 2022] GET /identity/v3/users/dddf1df5ba7d4b1c8c097547af0c29da => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:16.612246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdd5a7-68b1-4f57-a495-c2d0d39da954 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:16.707557 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:16.711788 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdd5a7-68b1-4f57-a495-c2d0d39da954 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:16.749067 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb3924-ef6b-4ca2-8fc9-13726a5273e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:16.749698 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb3924-ef6b-4ca2-8fc9-13726a5273e0 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:16.750014 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb3924-ef6b-4ca2-8fc9-13726a5273e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:16.750290 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb3924-ef6b-4ca2-8fc9-13726a5273e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:16.816736 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb3924-ef6b-4ca2-8fc9-13726a5273e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:16.818228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb3924-ef6b-4ca2-8fc9-13726a5273e0 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:16.915076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:16.920195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 567/1128] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:01:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:01:16.926237 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f0e37-3e21-40c8-b0f6-5eefb3f152f3 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:16.941786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:17.036938 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f0e37-3e21-40c8-b0f6-5eefb3f152f3 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:17.076826 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3686be-10b5-4e21-8594-57ad252c41bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:17.077442 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3686be-10b5-4e21-8594-57ad252c41bf None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:17.077740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3686be-10b5-4e21-8594-57ad252c41bf None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:17.078645 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3686be-10b5-4e21-8594-57ad252c41bf None admin] PATH_INFO: `/v3/projects/5b50615864e740e3a251e2af12ea3b06` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:17.080037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3686be-10b5-4e21-8594-57ad252c41bf None admin] RBAC: Authorizing `identity:get_project(project_id=5b50615864e740e3a251e2af12ea3b06)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:17.081628 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3686be-10b5-4e21-8594-57ad252c41bf None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:17.084565 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 562/1129] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 20:01:16 2022] GET /identity/v3/projects/5b50615864e740e3a251e2af12ea3b06 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:17.090080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb3924-ef6b-4ca2-8fc9-13726a5273e0 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:17.194056 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb3924-ef6b-4ca2-8fc9-13726a5273e0 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:17.228980 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e908ada-9305-41c3-9633-3731785e652e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:17.229579 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e908ada-9305-41c3-9633-3731785e652e None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:17.229894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e908ada-9305-41c3-9633-3731785e652e None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:17.230167 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e908ada-9305-41c3-9633-3731785e652e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:17.230883 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e908ada-9305-41c3-9633-3731785e652e None admin] RBAC: Adding query filter params (role.id=b9184599d95b4950beca34304cdd2eb8, scope.project.id=5b50615864e740e3a251e2af12ea3b06, user.id=dddf1df5ba7d4b1c8c097547af0c29da) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:17.231191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e908ada-9305-41c3-9633-3731785e652e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:17.232616 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e908ada-9305-41c3-9633-3731785e652e None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:17.238976 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 568/1130] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:01:17 2022] GET /identity/v3/role_assignments?user.id=dddf1df5ba7d4b1c8c097547af0c29da&scope.project.id=5b50615864e740e3a251e2af12ea3b06&role.id=b9184599d95b4950beca34304cdd2eb8 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:19.214096 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c382c0-6792-4e9e-a7d4-9eb983100c9e None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:19.214646 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c382c0-6792-4e9e-a7d4-9eb983100c9e None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:19.214646 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c382c0-6792-4e9e-a7d4-9eb983100c9e None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:19.215981 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 563/1131] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:01:19.221359 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d702bbf-2811-44a3-bae7-14c78db1e229 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:19.221846 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d702bbf-2811-44a3-bae7-14c78db1e229 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:19.221939 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d702bbf-2811-44a3-bae7-14c78db1e229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:19.240827 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-8d702bbf-2811-44a3-bae7-14c78db1e229 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:01:19.242064 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-8d702bbf-2811-44a3-bae7-14c78db1e229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:01:19.299806 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:19.311182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 569/1132] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:01:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:01:19.584871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c382c0-6792-4e9e-a7d4-9eb983100c9e None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:19.679544 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:19.700718 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c382c0-6792-4e9e-a7d4-9eb983100c9e None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:19.734800 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f72ea3-3813-4d2d-b746-602a5081bfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:19.735431 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f72ea3-3813-4d2d-b746-602a5081bfef None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:19.735703 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f72ea3-3813-4d2d-b746-602a5081bfef None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:19.735976 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f72ea3-3813-4d2d-b746-602a5081bfef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:19.737098 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f72ea3-3813-4d2d-b746-602a5081bfef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:19.738552 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f72ea3-3813-4d2d-b746-602a5081bfef None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:19.740130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 564/1133] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:01:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:19.746109 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d702bbf-2811-44a3-bae7-14c78db1e229 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:19.850033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d702bbf-2811-44a3-bae7-14c78db1e229 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:19.887455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c50fb-41e3-464d-bec6-d6c0a93ead6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:19.888065 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c50fb-41e3-464d-bec6-d6c0a93ead6b None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:19.888342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c50fb-41e3-464d-bec6-d6c0a93ead6b None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:19.888609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c50fb-41e3-464d-bec6-d6c0a93ead6b None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:19.889263 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c50fb-41e3-464d-bec6-d6c0a93ead6b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:19.891289 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c50fb-41e3-464d-bec6-d6c0a93ead6b None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:19.926080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 570/1134] 10.222.0.73 () {60 vars in 1141 bytes} [Mon Aug 29 20:01:19 2022] POST /identity/v3/projects => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:01:21.929719 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f00a2-8285-48dd-b07e-d04e9206886b None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:21.930280 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f00a2-8285-48dd-b07e-d04e9206886b None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:21.930280 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f00a2-8285-48dd-b07e-d04e9206886b None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:21.931703 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 565/1135] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:01:21.936924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee5dfb-a133-4021-adc7-b85e062bdd8a None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:21.937330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee5dfb-a133-4021-adc7-b85e062bdd8a None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:21.937569 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee5dfb-a133-4021-adc7-b85e062bdd8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:21.959952 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-61ee5dfb-a133-4021-adc7-b85e062bdd8a None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:01:21.961213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-61ee5dfb-a133-4021-adc7-b85e062bdd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:01:22.009003 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:22.041809 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 571/1136] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:01:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:01:22.376400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f00a2-8285-48dd-b07e-d04e9206886b None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:22.481436 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f00a2-8285-48dd-b07e-d04e9206886b None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:22.516117 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6106d-5d13-4e79-863f-a18a19439691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:22.516739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6106d-5d13-4e79-863f-a18a19439691 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:22.517022 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6106d-5d13-4e79-863f-a18a19439691 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:22.517298 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6106d-5d13-4e79-863f-a18a19439691 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:22.518435 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d6106d-5d13-4e79-863f-a18a19439691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:22.519864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d6106d-5d13-4e79-863f-a18a19439691 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:22.521480 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 566/1137] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:01:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:22.527302 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee5dfb-a133-4021-adc7-b85e062bdd8a None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:22.667958 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee5dfb-a133-4021-adc7-b85e062bdd8a None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:22.709589 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e7432d-e457-4d83-826d-c581f5a31c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:22.710663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7432d-e457-4d83-826d-c581f5a31c96 None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:22.711123 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7432d-e457-4d83-826d-c581f5a31c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:22.711604 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7432d-e457-4d83-826d-c581f5a31c96 None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:22.712806 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e7432d-e457-4d83-826d-c581f5a31c96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:22.715570 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e7432d-e457-4d83-826d-c581f5a31c96 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:22.742012 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 572/1138] 10.222.0.73 () {60 vars in 1132 bytes} [Mon Aug 29 20:01:22 2022] POST /identity/v3/users => generated 286 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:01:24.707205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b6ebf-e4b0-48d2-8992-9f98dd75e37b None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:24.707766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b6ebf-e4b0-48d2-8992-9f98dd75e37b None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:24.707766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b6ebf-e4b0-48d2-8992-9f98dd75e37b None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:24.709095 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 567/1139] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:01:24.715156 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c6c19-5cfd-4076-b123-6f013404229f None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:24.715606 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c6c19-5cfd-4076-b123-6f013404229f None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:24.715707 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c6c19-5cfd-4076-b123-6f013404229f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:24.735083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-ac3c6c19-5cfd-4076-b123-6f013404229f None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:01:24.736451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ac3c6c19-5cfd-4076-b123-6f013404229f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:01:24.763162 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:24.809375 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 573/1140] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:01:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:01:25.081429 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b6ebf-e4b0-48d2-8992-9f98dd75e37b None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:25.101943 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:25.185759 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b6ebf-e4b0-48d2-8992-9f98dd75e37b None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:25.219439 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8bd5d-3b8c-40d5-a3bc-937470e91595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:25.220076 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8bd5d-3b8c-40d5-a3bc-937470e91595 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:25.220355 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8bd5d-3b8c-40d5-a3bc-937470e91595 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:25.220623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8bd5d-3b8c-40d5-a3bc-937470e91595 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:25.230570 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec8bd5d-3b8c-40d5-a3bc-937470e91595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:01:25.235907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-eec8bd5d-3b8c-40d5-a3bc-937470e91595 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:01:25.235907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:01:25.235907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:01:25.235907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:01:25.235907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:01:25.235907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:01:25.235907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:01:25.235907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:01:25.235907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:01:25.235907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:01:25.235907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:01:25.235907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:01:25.235907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:01:25.235907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:01:25.235907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:01:25.235907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:01:25.237093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:01:25.237093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:01:25.237093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:01:25.237093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:01:25.237093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:01:25.237093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:01:25.237093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:01:25.237093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:01:25.237093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:01:25.237093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:01:25.237093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:01:25.237093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:01:25.237093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:01:25.237093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:01:25.237093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:01:25.237093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:01:25.238243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:01:25.238243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:01:25.238243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:01:25.238243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:01:25.238243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:01:25.238243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:01:25.238243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:01:25.238243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:01:25.238243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:01:25.238243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:01:25.238243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:01:25.238243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:01:25.238243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 568/1141] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 20:01:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:01:25.243134 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c6c19-5cfd-4076-b123-6f013404229f None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:25.340660 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c6c19-5cfd-4076-b123-6f013404229f None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:25.374263 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e3cea-4acb-4b3b-9590-5f59079b5657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:25.374861 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e3cea-4acb-4b3b-9590-5f59079b5657 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:25.375727 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e3cea-4acb-4b3b-9590-5f59079b5657 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:25.376004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e3cea-4acb-4b3b-9590-5f59079b5657 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:25.376667 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e3cea-4acb-4b3b-9590-5f59079b5657 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:25.376972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e3cea-4acb-4b3b-9590-5f59079b5657 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:25.378415 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e3cea-4acb-4b3b-9590-5f59079b5657 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:25.386900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 574/1142] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 20:01:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:25.393217 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8bd5d-3b8c-40d5-a3bc-937470e91595 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:25.477416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:25.508811 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8bd5d-3b8c-40d5-a3bc-937470e91595 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:25.543628 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace77ae-d11a-4c86-8a49-0cd965303ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:25.544257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace77ae-d11a-4c86-8a49-0cd965303ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:25.544537 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace77ae-d11a-4c86-8a49-0cd965303ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:25.544810 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace77ae-d11a-4c86-8a49-0cd965303ef4 None admin] PATH_INFO: `/v3/users/b6680f7ddd15430595b644b04e645fd9` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:25.558777 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace77ae-d11a-4c86-8a49-0cd965303ef4 None admin] RBAC: Authorizing `identity:get_user(user_id=b6680f7ddd15430595b644b04e645fd9)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:25.560213 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace77ae-d11a-4c86-8a49-0cd965303ef4 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:25.561879 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 569/1143] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 20:01:25 2022] GET /identity/v3/users/b6680f7ddd15430595b644b04e645fd9 => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:25.568076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e3cea-4acb-4b3b-9590-5f59079b5657 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:25.594569 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:25.678003 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e3cea-4acb-4b3b-9590-5f59079b5657 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:25.710502 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6bbe8-3958-4c57-b922-7cb62b62fe76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:25.711238 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6bbe8-3958-4c57-b922-7cb62b62fe76 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:25.711451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6bbe8-3958-4c57-b922-7cb62b62fe76 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:25.711847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6bbe8-3958-4c57-b922-7cb62b62fe76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:25.782359 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6bbe8-3958-4c57-b922-7cb62b62fe76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:25.784309 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6bbe8-3958-4c57-b922-7cb62b62fe76 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:25.807823 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:25.887594 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 575/1144] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:01:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:01:25.894714 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace77ae-d11a-4c86-8a49-0cd965303ef4 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:26.003833 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace77ae-d11a-4c86-8a49-0cd965303ef4 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:26.038184 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cb454-e3c8-440a-9ffe-b077e087e3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:26.038798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cb454-e3c8-440a-9ffe-b077e087e3be None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:26.039074 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cb454-e3c8-440a-9ffe-b077e087e3be None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:26.039341 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cb454-e3c8-440a-9ffe-b077e087e3be None admin] PATH_INFO: `/v3/projects/650a9a09913a4f5abea7faf2b558b7cb` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:26.040449 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77cb454-e3c8-440a-9ffe-b077e087e3be None admin] RBAC: Authorizing `identity:get_project(project_id=650a9a09913a4f5abea7faf2b558b7cb)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:26.041898 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77cb454-e3c8-440a-9ffe-b077e087e3be None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:26.043512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 570/1145] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 20:01:25 2022] GET /identity/v3/projects/650a9a09913a4f5abea7faf2b558b7cb => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:26.049957 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6bbe8-3958-4c57-b922-7cb62b62fe76 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:26.151217 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6bbe8-3958-4c57-b922-7cb62b62fe76 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:26.174443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:26.186148 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787963a-fdb1-4481-88e6-2dc897bca9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:26.186745 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787963a-fdb1-4481-88e6-2dc897bca9c9 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:26.187020 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787963a-fdb1-4481-88e6-2dc897bca9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:26.187289 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787963a-fdb1-4481-88e6-2dc897bca9c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:26.188026 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1787963a-fdb1-4481-88e6-2dc897bca9c9 None admin] RBAC: Adding query filter params (role.id=296ee2c194824485af71618619f82e09, scope.project.id=650a9a09913a4f5abea7faf2b558b7cb, user.id=b6680f7ddd15430595b644b04e645fd9) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:26.188331 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1787963a-fdb1-4481-88e6-2dc897bca9c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:26.189774 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1787963a-fdb1-4481-88e6-2dc897bca9c9 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:26.194986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 576/1146] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:01:26 2022] GET /identity/v3/role_assignments?user.id=b6680f7ddd15430595b644b04e645fd9&scope.project.id=650a9a09913a4f5abea7faf2b558b7cb&role.id=296ee2c194824485af71618619f82e09 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:28.086490 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def8e8e-4bb3-4bd0-b83c-9d69be8c58fc None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:28.087037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def8e8e-4bb3-4bd0-b83c-9d69be8c58fc None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:28.087037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def8e8e-4bb3-4bd0-b83c-9d69be8c58fc None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:28.088380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 571/1147] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:01:28.093539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a15d0c-cc1b-46ee-85cd-cedeed62e8f1 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:28.093983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a15d0c-cc1b-46ee-85cd-cedeed62e8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:28.094124 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a15d0c-cc1b-46ee-85cd-cedeed62e8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:28.112683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-87a15d0c-cc1b-46ee-85cd-cedeed62e8f1 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:01:28.113906 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-87a15d0c-cc1b-46ee-85cd-cedeed62e8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:01:28.180472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 577/1148] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:01:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:01:28.449975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def8e8e-4bb3-4bd0-b83c-9d69be8c58fc None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:28.453892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:28.557759 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def8e8e-4bb3-4bd0-b83c-9d69be8c58fc None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:28.591459 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cffcf4-9027-4390-b465-b1fa2a9481e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:28.592088 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cffcf4-9027-4390-b465-b1fa2a9481e3 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:28.592367 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cffcf4-9027-4390-b465-b1fa2a9481e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:28.592638 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cffcf4-9027-4390-b465-b1fa2a9481e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:28.603903 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cffcf4-9027-4390-b465-b1fa2a9481e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:01:28.608744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-19cffcf4-9027-4390-b465-b1fa2a9481e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:01:28.608744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:01:28.608744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:01:28.608744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:01:28.608744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:01:28.608744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:01:28.608744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:01:28.608744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:01:28.608744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:01:28.608744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:01:28.608744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:01:28.608744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:01:28.608744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:01:28.608744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:01:28.608744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:01:28.608744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:01:28.610582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:01:28.610582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:01:28.610582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:01:28.610582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:01:28.610582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:01:28.610582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:01:28.610582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:01:28.610582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:01:28.610582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:01:28.610582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:01:28.610582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:01:28.610582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:01:28.610582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:01:28.610582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:01:28.610582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:01:28.610582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:01:28.611954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:01:28.611954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:01:28.611954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:01:28.611954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:01:28.611954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:01:28.611954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:01:28.611954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:01:28.611954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:01:28.611954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:01:28.611954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:01:28.611954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:01:28.611954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:01:28.611954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 572/1149] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 20:01:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:01:28.616884 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a15d0c-cc1b-46ee-85cd-cedeed62e8f1 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:28.721941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a15d0c-cc1b-46ee-85cd-cedeed62e8f1 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:28.741530 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:28.760188 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a4043-d151-4df0-b8d6-31b773d31532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:28.760793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a4043-d151-4df0-b8d6-31b773d31532 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:28.761075 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a4043-d151-4df0-b8d6-31b773d31532 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:28.761345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a4043-d151-4df0-b8d6-31b773d31532 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:28.762038 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a4043-d151-4df0-b8d6-31b773d31532 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:28.762342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a4043-d151-4df0-b8d6-31b773d31532 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:28.763707 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a4043-d151-4df0-b8d6-31b773d31532 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:28.772451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 578/1150] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 20:01:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:28.778351 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cffcf4-9027-4390-b465-b1fa2a9481e3 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:28.840910 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:28.881168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cffcf4-9027-4390-b465-b1fa2a9481e3 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:28.915563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce453177-d367-4fde-a006-e9bda0c237eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:28.916184 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce453177-d367-4fde-a006-e9bda0c237eb None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:28.916460 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce453177-d367-4fde-a006-e9bda0c237eb None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:28.916738 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce453177-d367-4fde-a006-e9bda0c237eb None admin] PATH_INFO: `/v3/users/b6680f7ddd15430595b644b04e645fd9` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:28.917931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce453177-d367-4fde-a006-e9bda0c237eb None admin] RBAC: Authorizing `identity:get_user(user_id=b6680f7ddd15430595b644b04e645fd9)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:28.919353 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce453177-d367-4fde-a006-e9bda0c237eb None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:28.920919 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 573/1151] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 20:01:28 2022] GET /identity/v3/users/b6680f7ddd15430595b644b04e645fd9 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:28.926928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a4043-d151-4df0-b8d6-31b773d31532 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:29.028746 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a4043-d151-4df0-b8d6-31b773d31532 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:29.062672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368cb78-d3a9-4501-a0a2-4aaf8c4aa1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:29.063481 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368cb78-d3a9-4501-a0a2-4aaf8c4aa1fc None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:29.063760 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368cb78-d3a9-4501-a0a2-4aaf8c4aa1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:29.064037 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368cb78-d3a9-4501-a0a2-4aaf8c4aa1fc None admin] PATH_INFO: `/v3/projects/650a9a09913a4f5abea7faf2b558b7cb` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:29.065158 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9368cb78-d3a9-4501-a0a2-4aaf8c4aa1fc None admin] RBAC: Authorizing `identity:get_project(project_id=650a9a09913a4f5abea7faf2b558b7cb)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:29.066621 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9368cb78-d3a9-4501-a0a2-4aaf8c4aa1fc None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:29.068252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 579/1152] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 20:01:28 2022] GET /identity/v3/projects/650a9a09913a4f5abea7faf2b558b7cb => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:01:29.074246 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce453177-d367-4fde-a006-e9bda0c237eb None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:29.172825 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce453177-d367-4fde-a006-e9bda0c237eb None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:29.200884 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:29.206582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4fc377-3529-49ca-abc8-c3dee330a3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:29.207230 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fc377-3529-49ca-abc8-c3dee330a3eb None admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:29.207506 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fc377-3529-49ca-abc8-c3dee330a3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:29.207778 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fc377-3529-49ca-abc8-c3dee330a3eb None admin] PATH_INFO: `/v3/projects/650a9a09913a4f5abea7faf2b558b7cb/users/b6680f7ddd15430595b644b04e645fd9/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:29.209663 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4fc377-3529-49ca-abc8-c3dee330a3eb None admin] RBAC: Authorizing `identity:create_grant(project_id=650a9a09913a4f5abea7faf2b558b7cb, user_id=b6680f7ddd15430595b644b04e645fd9, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:29.211107 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4fc377-3529-49ca-abc8-c3dee330a3eb None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:29.224663 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 574/1153] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 20:01:29 2022] PUT /identity/v3/projects/650a9a09913a4f5abea7faf2b558b7cb/users/b6680f7ddd15430595b644b04e645fd9/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:01:31.152543 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003a37a-e25f-4273-9154-9a1e6eee7fe1 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:31.153089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003a37a-e25f-4273-9154-9a1e6eee7fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:31.153089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003a37a-e25f-4273-9154-9a1e6eee7fe1 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:31.154464 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 580/1154] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:01:31.160231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea397402-296f-4c03-9120-662b5745b812 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:31.160634 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea397402-296f-4c03-9120-662b5745b812 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:31.160775 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea397402-296f-4c03-9120-662b5745b812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:31.181603 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-ea397402-296f-4c03-9120-662b5745b812 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:01:31.183381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-ea397402-296f-4c03-9120-662b5745b812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:01:31.278767 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 575/1155] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:01:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:01:31.545954 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003a37a-e25f-4273-9154-9a1e6eee7fe1 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:31.594544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:31.648244 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003a37a-e25f-4273-9154-9a1e6eee7fe1 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:31.680790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149011f-d047-482e-a71c-60b516020d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:31.681403 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149011f-d047-482e-a71c-60b516020d3a None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:31.681681 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149011f-d047-482e-a71c-60b516020d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:31.681975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149011f-d047-482e-a71c-60b516020d3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:31.692361 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149011f-d047-482e-a71c-60b516020d3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:01:31.697070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-0149011f-d047-482e-a71c-60b516020d3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:01:31.697070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:01:31.697070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:01:31.697070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:01:31.697070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:01:31.697070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:01:31.697070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:01:31.697070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:01:31.697070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:01:31.697070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:01:31.697070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:01:31.697070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:01:31.697070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:01:31.697070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:01:31.697070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:01:31.697070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:01:31.698171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:01:31.698171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:01:31.698171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:01:31.698171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:01:31.698171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:01:31.698171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:01:31.698171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:01:31.698171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:01:31.698171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:01:31.698171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:01:31.698171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:01:31.698171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:01:31.698171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:01:31.698171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:01:31.698171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:01:31.698171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:01:31.699202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:01:31.699202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:01:31.699202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:01:31.699202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:01:31.699202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:01:31.699202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:01:31.699202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:01:31.699202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:01:31.699202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:01:31.699202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:01:31.699202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:01:31.699202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:01:31.699202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 581/1156] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 20:01:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:01:31.705064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea397402-296f-4c03-9120-662b5745b812 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:31.803963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea397402-296f-4c03-9120-662b5745b812 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:31.826747 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:31.840473 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d7b38c-ab21-4d99-853e-e3184073f815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:31.841086 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7b38c-ab21-4d99-853e-e3184073f815 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:31.841370 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7b38c-ab21-4d99-853e-e3184073f815 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:31.841643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7b38c-ab21-4d99-853e-e3184073f815 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:31.842347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d7b38c-ab21-4d99-853e-e3184073f815 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:31.842662 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d7b38c-ab21-4d99-853e-e3184073f815 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:31.844038 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d7b38c-ab21-4d99-853e-e3184073f815 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:31.852356 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 576/1157] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 20:01:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:31.858247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149011f-d047-482e-a71c-60b516020d3a None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:31.970753 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149011f-d047-482e-a71c-60b516020d3a None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:31.979185 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:32.011119 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc160c3-c32d-4d82-9c08-8d5c0a9dc514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:32.011760 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc160c3-c32d-4d82-9c08-8d5c0a9dc514 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:32.012044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc160c3-c32d-4d82-9c08-8d5c0a9dc514 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:32.012318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc160c3-c32d-4d82-9c08-8d5c0a9dc514 None admin] PATH_INFO: `/v3/users/b6680f7ddd15430595b644b04e645fd9` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:32.013473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc160c3-c32d-4d82-9c08-8d5c0a9dc514 None admin] RBAC: Authorizing `identity:get_user(user_id=b6680f7ddd15430595b644b04e645fd9)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:32.014941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc160c3-c32d-4d82-9c08-8d5c0a9dc514 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:32.016587 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 582/1158] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 20:01:31 2022] GET /identity/v3/users/b6680f7ddd15430595b644b04e645fd9 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:32.022621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d7b38c-ab21-4d99-853e-e3184073f815 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:32.130898 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d7b38c-ab21-4d99-853e-e3184073f815 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:32.164028 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d98d39-61ba-472c-afcb-2ca113d3f64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:32.164678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d98d39-61ba-472c-afcb-2ca113d3f64f None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:32.164889 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d98d39-61ba-472c-afcb-2ca113d3f64f None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:32.165159 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d98d39-61ba-472c-afcb-2ca113d3f64f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:32.218020 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:32.234403 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d98d39-61ba-472c-afcb-2ca113d3f64f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:32.236567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d98d39-61ba-472c-afcb-2ca113d3f64f None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:32.334290 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 577/1159] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:01:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:01:32.340366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc160c3-c32d-4d82-9c08-8d5c0a9dc514 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:32.440344 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc160c3-c32d-4d82-9c08-8d5c0a9dc514 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:32.473333 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7e0cd-c67a-4294-89ad-0c610160da92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:32.473978 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7e0cd-c67a-4294-89ad-0c610160da92 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:32.474261 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7e0cd-c67a-4294-89ad-0c610160da92 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:32.474540 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7e0cd-c67a-4294-89ad-0c610160da92 None admin] PATH_INFO: `/v3/projects/650a9a09913a4f5abea7faf2b558b7cb` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:32.475658 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb7e0cd-c67a-4294-89ad-0c610160da92 None admin] RBAC: Authorizing `identity:get_project(project_id=650a9a09913a4f5abea7faf2b558b7cb)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:32.477082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb7e0cd-c67a-4294-89ad-0c610160da92 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:32.478736 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 583/1160] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 20:01:32 2022] GET /identity/v3/projects/650a9a09913a4f5abea7faf2b558b7cb => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:32.485576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d98d39-61ba-472c-afcb-2ca113d3f64f None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:32.567894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:32.583987 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d98d39-61ba-472c-afcb-2ca113d3f64f None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:32.617503 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e83c92-badb-425b-9cb9-5b1ec447abaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:32.618058 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e83c92-badb-425b-9cb9-5b1ec447abaa None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:32.618340 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e83c92-badb-425b-9cb9-5b1ec447abaa None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:32.618611 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e83c92-badb-425b-9cb9-5b1ec447abaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:32.619318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e83c92-badb-425b-9cb9-5b1ec447abaa None admin] RBAC: Adding query filter params (role.id=296ee2c194824485af71618619f82e09, scope.project.id=650a9a09913a4f5abea7faf2b558b7cb, user.id=b6680f7ddd15430595b644b04e645fd9) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:32.619624 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e83c92-badb-425b-9cb9-5b1ec447abaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:32.621030 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e83c92-badb-425b-9cb9-5b1ec447abaa None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:32.626766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 578/1161] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:01:32 2022] GET /identity/v3/role_assignments?user.id=b6680f7ddd15430595b644b04e645fd9&scope.project.id=650a9a09913a4f5abea7faf2b558b7cb&role.id=296ee2c194824485af71618619f82e09 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:34.564325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346c64a-7fc0-4665-abd2-52d7752e9b72 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:34.564824 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346c64a-7fc0-4665-abd2-52d7752e9b72 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:34.564824 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346c64a-7fc0-4665-abd2-52d7752e9b72 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:34.566164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 584/1162] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:01:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:01:34.571323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f518b-c403-4bda-9b05-f2a8bddc8459 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:34.571728 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f518b-c403-4bda-9b05-f2a8bddc8459 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:34.571863 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f518b-c403-4bda-9b05-f2a8bddc8459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:34.590672 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-108f518b-c403-4bda-9b05-f2a8bddc8459 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:01:34.591920 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-108f518b-c403-4bda-9b05-f2a8bddc8459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:01:34.656576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 579/1163] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:01:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:01:34.930930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346c64a-7fc0-4665-abd2-52d7752e9b72 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:34.974866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:35.035164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346c64a-7fc0-4665-abd2-52d7752e9b72 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:35.076110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b24f91-f75a-4f31-8520-d1fe1e2c11f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:35.076939 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b24f91-f75a-4f31-8520-d1fe1e2c11f4 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:35.077090 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b24f91-f75a-4f31-8520-d1fe1e2c11f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:35.077362 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b24f91-f75a-4f31-8520-d1fe1e2c11f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:35.146074 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b24f91-f75a-4f31-8520-d1fe1e2c11f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:35.147521 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b24f91-f75a-4f31-8520-d1fe1e2c11f4 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:35.190070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:35.247693 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 585/1164] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:01:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:01:35.254331 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108f518b-c403-4bda-9b05-f2a8bddc8459 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:35.328936 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:35.350380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108f518b-c403-4bda-9b05-f2a8bddc8459 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:35.383794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc6a8e-85e7-4bc1-9093-b9b39013bfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:35.384427 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc6a8e-85e7-4bc1-9093-b9b39013bfca None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:35.384706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc6a8e-85e7-4bc1-9093-b9b39013bfca None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:35.384976 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc6a8e-85e7-4bc1-9093-b9b39013bfca None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:35.391577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc6a8e-85e7-4bc1-9093-b9b39013bfca None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:35.393005 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc6a8e-85e7-4bc1-9093-b9b39013bfca None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:35.404136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-52bc6a8e-85e7-4bc1-9093-b9b39013bfca None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 29 20:01:35.404136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:01:35.404136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 29 20:01:35.404136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 29 20:01:35.404136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:01:35.404136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:01:35.404136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:01:35.404136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:01:35.404136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 29 20:01:35.404136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:01:35.404136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:01:35.404136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:01:35.404136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:01:35.404136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:01:35.404136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:01:35.404136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:01:35.404136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:01:35.405347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:01:35.405347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:01:35.405347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:01:35.405347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:01:35.405347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:01:35.405347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:01:35.405347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:01:35.405347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:01:35.405347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:01:35.405347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:01:35.405347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:01:35.405347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:01:35.405347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:01:35.405347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:01:35.405347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:01:35.405347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:01:35.407165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:01:35.407165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:01:35.407165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:01:35.407165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:01:35.407165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:01:35.407165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:01:35.407165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:01:35.407165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:01:35.407165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:01:35.407165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:01:35.407165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 29 20:01:35.407165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:01:35.407165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 29 20:01:35.407165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:01:35.407165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 580/1165] 10.222.0.73 () {56 vars in 1117 bytes} [Mon Aug 29 20:01:35 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:01:35.411777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b24f91-f75a-4f31-8520-d1fe1e2c11f4 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:35.508752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b24f91-f75a-4f31-8520-d1fe1e2c11f4 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:35.542813 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce01257-929a-47df-b6b8-a6fbf3299cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:35.543414 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce01257-929a-47df-b6b8-a6fbf3299cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:35.543694 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce01257-929a-47df-b6b8-a6fbf3299cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:35.543968 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce01257-929a-47df-b6b8-a6fbf3299cf2 None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:35.544599 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce01257-929a-47df-b6b8-a6fbf3299cf2 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:35.544902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce01257-929a-47df-b6b8-a6fbf3299cf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:35.546301 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce01257-929a-47df-b6b8-a6fbf3299cf2 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:35.554153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 586/1166] 10.222.0.73 () {56 vars in 1115 bytes} [Mon Aug 29 20:01:35 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:37.469344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c33c5-34d7-4f72-8f7e-a45cee48921f None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:37.470357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c33c5-34d7-4f72-8f7e-a45cee48921f None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:37.470508 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c33c5-34d7-4f72-8f7e-a45cee48921f None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:37.471851 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 581/1167] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:01:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:01:37.476931 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebad8a0-1cf0-4fe1-8991-2c9956a06049 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:37.477323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebad8a0-1cf0-4fe1-8991-2c9956a06049 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:37.477486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebad8a0-1cf0-4fe1-8991-2c9956a06049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:37.490669 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:37.496185 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-bebad8a0-1cf0-4fe1-8991-2c9956a06049 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:01:37.497356 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-bebad8a0-1cf0-4fe1-8991-2c9956a06049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:01:37.564001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 587/1168] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:01:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:01:37.835252 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c33c5-34d7-4f72-8f7e-a45cee48921f None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:37.939101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c33c5-34d7-4f72-8f7e-a45cee48921f None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:37.971742 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a99a59-be20-4870-b366-aed356d1987c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:37.972345 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a99a59-be20-4870-b366-aed356d1987c None admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:37.972623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a99a59-be20-4870-b366-aed356d1987c None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:37.972895 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a99a59-be20-4870-b366-aed356d1987c None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:37.973555 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a99a59-be20-4870-b366-aed356d1987c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:37.974940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a99a59-be20-4870-b366-aed356d1987c None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:37.984553 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:37.995033 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 582/1169] 10.222.0.73 () {60 vars in 1138 bytes} [Mon Aug 29 20:01:37 2022] POST /identity/v3/domains => generated 267 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:01:39.913078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07190bfb-4fde-467b-bb21-77e895f5dcd0 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:39.913607 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07190bfb-4fde-467b-bb21-77e895f5dcd0 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:39.913923 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07190bfb-4fde-467b-bb21-77e895f5dcd0 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:39.915290 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 588/1170] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:01:39.920273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295093d-4d23-4089-ba98-1486e21a4253 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:39.920673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295093d-4d23-4089-ba98-1486e21a4253 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:39.920816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295093d-4d23-4089-ba98-1486e21a4253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:39.939545 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-f295093d-4d23-4089-ba98-1486e21a4253 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:01:39.940777 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-f295093d-4d23-4089-ba98-1486e21a4253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:01:40.008352 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 583/1171] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:01:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:01:40.275970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07190bfb-4fde-467b-bb21-77e895f5dcd0 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:40.378524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07190bfb-4fde-467b-bb21-77e895f5dcd0 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:40.412355 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aaef47-64f9-4123-af94-2cb40faee1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:40.412987 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aaef47-64f9-4123-af94-2cb40faee1bf None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:40.413270 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aaef47-64f9-4123-af94-2cb40faee1bf None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:40.413546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aaef47-64f9-4123-af94-2cb40faee1bf None admin] PATH_INFO: `/v3/domains/edd62bb78f064aac96933b98430e1492` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:40.416219 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:40.425193 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aaef47-64f9-4123-af94-2cb40faee1bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=edd62bb78f064aac96933b98430e1492)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:40.426801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aaef47-64f9-4123-af94-2cb40faee1bf None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:40.428450 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 589/1172] 10.222.0.73 () {56 vars in 1183 bytes} [Mon Aug 29 20:01:40 2022] GET /identity/v3/domains/edd62bb78f064aac96933b98430e1492 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:40.434437 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295093d-4d23-4089-ba98-1486e21a4253 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:40.533664 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295093d-4d23-4089-ba98-1486e21a4253 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:40.559084 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:40.568107 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a497b-71bf-45ac-bd2d-7380da571158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:40.568707 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a497b-71bf-45ac-bd2d-7380da571158 None admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:40.568980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a497b-71bf-45ac-bd2d-7380da571158 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:40.569244 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a497b-71bf-45ac-bd2d-7380da571158 None admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:40.570076 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a497b-71bf-45ac-bd2d-7380da571158 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:40.571525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a497b-71bf-45ac-bd2d-7380da571158 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:40.605432 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 584/1173] 10.222.0.73 () {60 vars in 1141 bytes} [Mon Aug 29 20:01:40 2022] POST /identity/v3/projects => generated 361 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:01:42.544362 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b2ea2-d479-4f2d-850f-318810d789af None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:42.544879 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b2ea2-d479-4f2d-850f-318810d789af None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:42.544879 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b2ea2-d479-4f2d-850f-318810d789af None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:42.546197 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 590/1174] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:01:42.551982 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e897a5-6dc7-4ab4-b09b-d588348776f1 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:42.552379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e897a5-6dc7-4ab4-b09b-d588348776f1 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:42.552535 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e897a5-6dc7-4ab4-b09b-d588348776f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:42.571376 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-47e897a5-6dc7-4ab4-b09b-d588348776f1 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:01:42.572582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-47e897a5-6dc7-4ab4-b09b-d588348776f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:01:42.638743 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 585/1175] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:01:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:01:42.910985 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b2ea2-d479-4f2d-850f-318810d789af None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:43.010881 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b2ea2-d479-4f2d-850f-318810d789af None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:43.044067 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a5ff3-68c4-4552-afb1-f267d1507be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:43.044690 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a5ff3-68c4-4552-afb1-f267d1507be1 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:43.044963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a5ff3-68c4-4552-afb1-f267d1507be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:43.045228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a5ff3-68c4-4552-afb1-f267d1507be1 None admin] PATH_INFO: `/v3/domains/edd62bb78f064aac96933b98430e1492` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:43.046452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9a5ff3-68c4-4552-afb1-f267d1507be1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=edd62bb78f064aac96933b98430e1492)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:43.048621 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9a5ff3-68c4-4552-afb1-f267d1507be1 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:43.050562 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 591/1176] 10.222.0.73 () {56 vars in 1183 bytes} [Mon Aug 29 20:01:42 2022] GET /identity/v3/domains/edd62bb78f064aac96933b98430e1492 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:43.056397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e897a5-6dc7-4ab4-b09b-d588348776f1 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:43.156444 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e897a5-6dc7-4ab4-b09b-d588348776f1 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:43.167257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:43.192981 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69132615-ef86-4b1f-9ecb-1e55f8d7e247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:43.193587 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69132615-ef86-4b1f-9ecb-1e55f8d7e247 None admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:43.193902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69132615-ef86-4b1f-9ecb-1e55f8d7e247 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:43.194178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69132615-ef86-4b1f-9ecb-1e55f8d7e247 None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:43.194818 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69132615-ef86-4b1f-9ecb-1e55f8d7e247 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:43.196243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69132615-ef86-4b1f-9ecb-1e55f8d7e247 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:43.217316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 586/1177] 10.222.0.73 () {60 vars in 1132 bytes} [Mon Aug 29 20:01:43 2022] POST /identity/v3/users => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:01:45.192633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd524c-82f9-4a39-84d8-762e7aef0667 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:45.193175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd524c-82f9-4a39-84d8-762e7aef0667 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:45.193175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd524c-82f9-4a39-84d8-762e7aef0667 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:45.194546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 592/1178] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:01:45.200010 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793235c-3dcd-4119-9e9f-ba3f62a74459 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:45.200443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793235c-3dcd-4119-9e9f-ba3f62a74459 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:45.200562 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793235c-3dcd-4119-9e9f-ba3f62a74459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:45.219567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-5793235c-3dcd-4119-9e9f-ba3f62a74459 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:01:45.220868 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-5793235c-3dcd-4119-9e9f-ba3f62a74459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:01:45.287531 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 587/1179] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:01:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:01:45.554099 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd524c-82f9-4a39-84d8-762e7aef0667 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:45.616854 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:45.655329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd524c-82f9-4a39-84d8-762e7aef0667 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:45.689895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ef2bd-e4f3-412a-a71e-156d7f4a4619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:45.690516 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ef2bd-e4f3-412a-a71e-156d7f4a4619 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:45.690795 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ef2bd-e4f3-412a-a71e-156d7f4a4619 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:45.691065 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ef2bd-e4f3-412a-a71e-156d7f4a4619 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:45.701048 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ef2bd-e4f3-412a-a71e-156d7f4a4619 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:01:45.706080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-c97ef2bd-e4f3-412a-a71e-156d7f4a4619 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:01:45.706080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:01:45.706080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:01:45.706080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:01:45.706080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:01:45.706080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:01:45.706080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:01:45.706080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:01:45.706080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:01:45.706080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:01:45.706080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:01:45.706080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:01:45.706080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:01:45.706080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:01:45.706080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:01:45.706080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:01:45.707200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:01:45.707200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:01:45.707200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:01:45.707200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:01:45.707200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:01:45.707200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:01:45.707200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:01:45.707200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:01:45.707200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:01:45.707200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:01:45.707200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:01:45.707200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:01:45.707200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:01:45.707200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:01:45.707200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:01:45.707200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:01:45.708218 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:01:45.708218 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:01:45.708218 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:01:45.708218 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:01:45.708218 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:01:45.708218 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:01:45.708218 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:01:45.708218 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:01:45.708218 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:01:45.708218 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:01:45.708218 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:01:45.708218 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:01:45.708218 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 593/1180] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 20:01:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:01:45.713243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793235c-3dcd-4119-9e9f-ba3f62a74459 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:45.807952 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:45.814179 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793235c-3dcd-4119-9e9f-ba3f62a74459 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:45.848215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e773d77-ecf9-46c4-88be-f56a76f998d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:45.848896 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e773d77-ecf9-46c4-88be-f56a76f998d7 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:45.849103 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e773d77-ecf9-46c4-88be-f56a76f998d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:45.849374 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e773d77-ecf9-46c4-88be-f56a76f998d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:45.850068 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e773d77-ecf9-46c4-88be-f56a76f998d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:45.850368 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e773d77-ecf9-46c4-88be-f56a76f998d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:45.851734 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e773d77-ecf9-46c4-88be-f56a76f998d7 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:45.860126 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 588/1181] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 20:01:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:45.867158 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ef2bd-e4f3-412a-a71e-156d7f4a4619 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:45.970981 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ef2bd-e4f3-412a-a71e-156d7f4a4619 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:45.994935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:46.006553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b2251-a819-4ba9-9707-88ea4f42a50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:46.007185 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b2251-a819-4ba9-9707-88ea4f42a50c None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:46.007467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b2251-a819-4ba9-9707-88ea4f42a50c None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:46.007744 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b2251-a819-4ba9-9707-88ea4f42a50c None admin] PATH_INFO: `/v3/users/995fb744cc9d49a28440be30b52ab5c8` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:46.021217 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b2251-a819-4ba9-9707-88ea4f42a50c None admin] RBAC: Authorizing `identity:get_user(user_id=995fb744cc9d49a28440be30b52ab5c8)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:46.022730 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b2251-a819-4ba9-9707-88ea4f42a50c None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:46.024366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 594/1182] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 20:01:45 2022] GET /identity/v3/users/995fb744cc9d49a28440be30b52ab5c8 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:01:46.030390 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e773d77-ecf9-46c4-88be-f56a76f998d7 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:46.131857 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e773d77-ecf9-46c4-88be-f56a76f998d7 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:46.166893 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34b220-f0a7-4ba1-bfb6-ee62724e27d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:46.167499 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34b220-f0a7-4ba1-bfb6-ee62724e27d5 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:46.167781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34b220-f0a7-4ba1-bfb6-ee62724e27d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:46.168052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34b220-f0a7-4ba1-bfb6-ee62724e27d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:46.196034 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:46.236325 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34b220-f0a7-4ba1-bfb6-ee62724e27d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:46.237802 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34b220-f0a7-4ba1-bfb6-ee62724e27d5 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:46.333982 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 589/1183] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:01:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:01:46.340320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b2251-a819-4ba9-9707-88ea4f42a50c None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:46.439049 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b2251-a819-4ba9-9707-88ea4f42a50c None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:46.473378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242bb4fb-11cc-42e8-ae0f-b8f9faf29f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:46.474029 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bb4fb-11cc-42e8-ae0f-b8f9faf29f72 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:46.474311 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bb4fb-11cc-42e8-ae0f-b8f9faf29f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:46.474586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bb4fb-11cc-42e8-ae0f-b8f9faf29f72 None admin] PATH_INFO: `/v3/projects/14bcb300457143cdb896128ab4390384` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:46.475710 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242bb4fb-11cc-42e8-ae0f-b8f9faf29f72 None admin] RBAC: Authorizing `identity:get_project(project_id=14bcb300457143cdb896128ab4390384)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:46.477144 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242bb4fb-11cc-42e8-ae0f-b8f9faf29f72 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:46.478807 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 595/1184] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 20:01:46 2022] GET /identity/v3/projects/14bcb300457143cdb896128ab4390384 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:46.484916 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34b220-f0a7-4ba1-bfb6-ee62724e27d5 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:46.541618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:46.583049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34b220-f0a7-4ba1-bfb6-ee62724e27d5 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:46.615973 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafbe506-2d33-4e96-aacc-8692dc563d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:46.616498 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbe506-2d33-4e96-aacc-8692dc563d09 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:46.616779 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbe506-2d33-4e96-aacc-8692dc563d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:46.617049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbe506-2d33-4e96-aacc-8692dc563d09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:46.617801 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafbe506-2d33-4e96-aacc-8692dc563d09 None admin] RBAC: Adding query filter params (role.id=296ee2c194824485af71618619f82e09, scope.project.id=14bcb300457143cdb896128ab4390384, user.id=995fb744cc9d49a28440be30b52ab5c8) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:46.618148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafbe506-2d33-4e96-aacc-8692dc563d09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:46.619559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafbe506-2d33-4e96-aacc-8692dc563d09 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:46.625059 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 590/1185] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:01:46 2022] GET /identity/v3/role_assignments?user.id=995fb744cc9d49a28440be30b52ab5c8&scope.project.id=14bcb300457143cdb896128ab4390384&role.id=296ee2c194824485af71618619f82e09 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:48.531641 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5605c22-51f4-458c-85f5-5eee2452db59 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:48.532140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5605c22-51f4-458c-85f5-5eee2452db59 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:48.532140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5605c22-51f4-458c-85f5-5eee2452db59 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:48.533477 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 596/1186] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:01:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:01:48.538914 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd3cb4-3c1b-4151-9bef-f57d1b2b3e00 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:48.539297 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd3cb4-3c1b-4151-9bef-f57d1b2b3e00 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:48.539470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd3cb4-3c1b-4151-9bef-f57d1b2b3e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:48.558160 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-13fd3cb4-3c1b-4151-9bef-f57d1b2b3e00 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:01:48.559356 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-13fd3cb4-3c1b-4151-9bef-f57d1b2b3e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:01:48.624021 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 591/1187] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:01:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:01:48.896311 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5605c22-51f4-458c-85f5-5eee2452db59 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:48.945981 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:48.994918 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5605c22-51f4-458c-85f5-5eee2452db59 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:49.029153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef5b44-3928-4d84-b515-5d42c3e6e639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:49.029816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef5b44-3928-4d84-b515-5d42c3e6e639 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:49.030097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef5b44-3928-4d84-b515-5d42c3e6e639 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:49.030366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef5b44-3928-4d84-b515-5d42c3e6e639 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:49.040211 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef5b44-3928-4d84-b515-5d42c3e6e639 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:01:49.044979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-97ef5b44-3928-4d84-b515-5d42c3e6e639 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:01:49.044979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:01:49.044979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:01:49.044979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:01:49.044979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:01:49.044979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:01:49.044979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:01:49.044979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:01:49.044979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:01:49.044979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:01:49.044979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:01:49.044979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:01:49.044979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:01:49.044979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:01:49.044979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:01:49.044979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:01:49.046102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:01:49.046102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:01:49.046102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:01:49.046102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:01:49.046102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:01:49.046102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:01:49.046102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:01:49.046102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:01:49.046102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:01:49.046102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:01:49.046102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:01:49.046102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:01:49.046102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:01:49.046102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:01:49.046102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:01:49.046102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:01:49.047015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:01:49.047015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:01:49.047015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:01:49.047015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:01:49.047015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:01:49.047015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:01:49.047015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:01:49.047015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:01:49.047015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:01:49.047015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:01:49.047015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:01:49.047015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:01:49.047015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 597/1188] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 20:01:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:01:49.052305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd3cb4-3c1b-4151-9bef-f57d1b2b3e00 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:49.107109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:49.154576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd3cb4-3c1b-4151-9bef-f57d1b2b3e00 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:49.187757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c204b-cbf1-43b2-a957-741e1b90196d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:49.188359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c204b-cbf1-43b2-a957-741e1b90196d None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:49.189206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c204b-cbf1-43b2-a957-741e1b90196d None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:49.189483 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c204b-cbf1-43b2-a957-741e1b90196d None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:49.190176 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c204b-cbf1-43b2-a957-741e1b90196d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:49.190483 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c204b-cbf1-43b2-a957-741e1b90196d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:49.191851 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c204b-cbf1-43b2-a957-741e1b90196d None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:49.199707 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 592/1189] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 20:01:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:49.206648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef5b44-3928-4d84-b515-5d42c3e6e639 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:49.306116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef5b44-3928-4d84-b515-5d42c3e6e639 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:49.317126 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:49.341006 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c57e15-d4c6-405e-9558-ccba75038af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:49.341623 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c57e15-d4c6-405e-9558-ccba75038af5 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:49.341931 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c57e15-d4c6-405e-9558-ccba75038af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:49.342201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c57e15-d4c6-405e-9558-ccba75038af5 None admin] PATH_INFO: `/v3/users/995fb744cc9d49a28440be30b52ab5c8` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:49.343320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c57e15-d4c6-405e-9558-ccba75038af5 None admin] RBAC: Authorizing `identity:get_user(user_id=995fb744cc9d49a28440be30b52ab5c8)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:49.344785 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c57e15-d4c6-405e-9558-ccba75038af5 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:49.346569 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 598/1190] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 20:01:49 2022] GET /identity/v3/users/995fb744cc9d49a28440be30b52ab5c8 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:49.352597 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c204b-cbf1-43b2-a957-741e1b90196d None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:49.457479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c204b-cbf1-43b2-a957-741e1b90196d None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:49.470783 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:49.491001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc0a17-61cc-4b5d-ba82-d432095f79f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:49.491624 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc0a17-61cc-4b5d-ba82-d432095f79f3 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:49.491911 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc0a17-61cc-4b5d-ba82-d432095f79f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:49.492184 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc0a17-61cc-4b5d-ba82-d432095f79f3 None admin] PATH_INFO: `/v3/projects/14bcb300457143cdb896128ab4390384` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:49.493297 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc0a17-61cc-4b5d-ba82-d432095f79f3 None admin] RBAC: Authorizing `identity:get_project(project_id=14bcb300457143cdb896128ab4390384)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:49.494762 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc0a17-61cc-4b5d-ba82-d432095f79f3 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:49.496391 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 593/1191] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 20:01:49 2022] GET /identity/v3/projects/14bcb300457143cdb896128ab4390384 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:49.503549 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c57e15-d4c6-405e-9558-ccba75038af5 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:49.602112 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c57e15-d4c6-405e-9558-ccba75038af5 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:49.635591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309107f-6d29-4bb6-8e53-a420c85a1d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:49.636227 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309107f-6d29-4bb6-8e53-a420c85a1d15 None admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:49.636501 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309107f-6d29-4bb6-8e53-a420c85a1d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:49.636769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309107f-6d29-4bb6-8e53-a420c85a1d15 None admin] PATH_INFO: `/v3/projects/14bcb300457143cdb896128ab4390384/users/995fb744cc9d49a28440be30b52ab5c8/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:49.638653 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7309107f-6d29-4bb6-8e53-a420c85a1d15 None admin] RBAC: Authorizing `identity:create_grant(project_id=14bcb300457143cdb896128ab4390384, user_id=995fb744cc9d49a28440be30b52ab5c8, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:49.640059 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7309107f-6d29-4bb6-8e53-a420c85a1d15 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:49.653272 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 599/1192] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 20:01:49 2022] PUT /identity/v3/projects/14bcb300457143cdb896128ab4390384/users/995fb744cc9d49a28440be30b52ab5c8/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:01:51.538786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805a71d-0e2c-48ad-a49e-b33e1dfe20fa None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:51.539357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805a71d-0e2c-48ad-a49e-b33e1dfe20fa None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:51.539357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805a71d-0e2c-48ad-a49e-b33e1dfe20fa None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:51.540565 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 594/1193] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:01:51.545877 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb80aa-83e4-4583-9ed4-75611b2b50d2 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:51.546309 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb80aa-83e4-4583-9ed4-75611b2b50d2 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:51.546434 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb80aa-83e4-4583-9ed4-75611b2b50d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:51.565971 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-7cbb80aa-83e4-4583-9ed4-75611b2b50d2 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:01:51.567159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-7cbb80aa-83e4-4583-9ed4-75611b2b50d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:01:51.596393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:51.635477 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 600/1194] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:01:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:01:51.906417 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805a71d-0e2c-48ad-a49e-b33e1dfe20fa None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:52.010860 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805a71d-0e2c-48ad-a49e-b33e1dfe20fa None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:52.044137 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8eb96f-1e2b-464d-b4bd-973dc161e53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:52.044767 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8eb96f-1e2b-464d-b4bd-973dc161e53f None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:52.045046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8eb96f-1e2b-464d-b4bd-973dc161e53f None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:52.045317 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8eb96f-1e2b-464d-b4bd-973dc161e53f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:52.056545 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8eb96f-1e2b-464d-b4bd-973dc161e53f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:01:52.061313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-4c8eb96f-1e2b-464d-b4bd-973dc161e53f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:01:52.061313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:01:52.061313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:01:52.061313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:01:52.061313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:01:52.061313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:01:52.061313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:01:52.061313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:01:52.061313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:01:52.061313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:01:52.061313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:01:52.061313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:01:52.061313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:01:52.061313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:01:52.061313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:01:52.061313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:01:52.062502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:01:52.062502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:01:52.062502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:01:52.062502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:01:52.062502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:01:52.062502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:01:52.062502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:01:52.062502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:01:52.062502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:01:52.062502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:01:52.062502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:01:52.062502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:01:52.062502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:01:52.062502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:01:52.062502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:01:52.062502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:01:52.063925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:01:52.063925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:01:52.063925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:01:52.063925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:01:52.063925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:01:52.063925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:01:52.063925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:01:52.063925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:01:52.063925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:01:52.063925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:01:52.063925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:01:52.063925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:01:52.063925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 595/1195] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 20:01:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:01:52.068528 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb80aa-83e4-4583-9ed4-75611b2b50d2 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:52.169987 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb80aa-83e4-4583-9ed4-75611b2b50d2 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:52.203147 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a12f4-4b52-4205-9d6e-398c04adc13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:52.203744 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a12f4-4b52-4205-9d6e-398c04adc13d None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:52.204026 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a12f4-4b52-4205-9d6e-398c04adc13d None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:52.204869 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a12f4-4b52-4205-9d6e-398c04adc13d None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:52.205524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a12f4-4b52-4205-9d6e-398c04adc13d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:52.205857 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a12f4-4b52-4205-9d6e-398c04adc13d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:52.207225 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a12f4-4b52-4205-9d6e-398c04adc13d None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:52.215384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 601/1196] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 20:01:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:52.222044 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8eb96f-1e2b-464d-b4bd-973dc161e53f None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:52.265519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:52.323547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8eb96f-1e2b-464d-b4bd-973dc161e53f None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:52.357596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645d7f7-7121-447d-b4c2-e73aa4878fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:52.358107 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645d7f7-7121-447d-b4c2-e73aa4878fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:52.358390 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645d7f7-7121-447d-b4c2-e73aa4878fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:52.358660 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645d7f7-7121-447d-b4c2-e73aa4878fb2 None admin] PATH_INFO: `/v3/users/995fb744cc9d49a28440be30b52ab5c8` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:52.359804 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a645d7f7-7121-447d-b4c2-e73aa4878fb2 None admin] RBAC: Authorizing `identity:get_user(user_id=995fb744cc9d49a28440be30b52ab5c8)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:52.361252 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a645d7f7-7121-447d-b4c2-e73aa4878fb2 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:52.362874 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 596/1197] 10.222.0.73 () {56 vars in 1177 bytes} [Mon Aug 29 20:01:52 2022] GET /identity/v3/users/995fb744cc9d49a28440be30b52ab5c8 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:52.368880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a12f4-4b52-4205-9d6e-398c04adc13d None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:52.401131 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:52.471482 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a12f4-4b52-4205-9d6e-398c04adc13d None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:52.504945 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01813dd5-9110-4bf8-b01b-4dde977c8504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:52.505574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01813dd5-9110-4bf8-b01b-4dde977c8504 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:52.505888 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01813dd5-9110-4bf8-b01b-4dde977c8504 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:52.506161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01813dd5-9110-4bf8-b01b-4dde977c8504 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:52.572165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01813dd5-9110-4bf8-b01b-4dde977c8504 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:52.573611 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01813dd5-9110-4bf8-b01b-4dde977c8504 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:52.602380 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:52.671494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 602/1198] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:01:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 20:01:52.677488 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645d7f7-7121-447d-b4c2-e73aa4878fb2 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:52.775563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645d7f7-7121-447d-b4c2-e73aa4878fb2 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:52.785492 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:52.809072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ffc065-1394-47ee-921f-b0a09a7f991a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:52.809723 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffc065-1394-47ee-921f-b0a09a7f991a None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:52.810011 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffc065-1394-47ee-921f-b0a09a7f991a None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:52.810287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffc065-1394-47ee-921f-b0a09a7f991a None admin] PATH_INFO: `/v3/projects/14bcb300457143cdb896128ab4390384` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:52.811410 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ffc065-1394-47ee-921f-b0a09a7f991a None admin] RBAC: Authorizing `identity:get_project(project_id=14bcb300457143cdb896128ab4390384)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:52.812842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ffc065-1394-47ee-921f-b0a09a7f991a None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:52.815297 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 597/1199] 10.222.0.73 () {56 vars in 1186 bytes} [Mon Aug 29 20:01:52 2022] GET /identity/v3/projects/14bcb300457143cdb896128ab4390384 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:01:52.821435 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01813dd5-9110-4bf8-b01b-4dde977c8504 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:01:52.921234 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01813dd5-9110-4bf8-b01b-4dde977c8504 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:01:52.950532 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:01:52.956912 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f60df-4d83-4680-a07f-60400107f0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:01:52.957513 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f60df-4d83-4680-a07f-60400107f0d2 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:01:52.957816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f60df-4d83-4680-a07f-60400107f0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:01:52.958092 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f60df-4d83-4680-a07f-60400107f0d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:01:52.958813 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f60df-4d83-4680-a07f-60400107f0d2 None admin] RBAC: Adding query filter params (role.id=296ee2c194824485af71618619f82e09, scope.project.id=14bcb300457143cdb896128ab4390384, user.id=995fb744cc9d49a28440be30b52ab5c8) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:01:52.959121 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f60df-4d83-4680-a07f-60400107f0d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:01:52.960553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f60df-4d83-4680-a07f-60400107f0d2 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:01:52.966314 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 603/1200] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:01:52 2022] GET /identity/v3/role_assignments?user.id=995fb744cc9d49a28440be30b52ab5c8&scope.project.id=14bcb300457143cdb896128ab4390384&role.id=296ee2c194824485af71618619f82e09 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:02:02.804961 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=11 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:02:02.963751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:02:49.172108 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e990d-78dc-4471-92cf-4c9f769c1696 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:49.172687 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e990d-78dc-4471-92cf-4c9f769c1696 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:49.172687 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e990d-78dc-4471-92cf-4c9f769c1696 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:49.174126 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 598/1201] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:02:49.180570 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdee2d-3dbe-4800-99dc-72248084475b None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:49.181107 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdee2d-3dbe-4800-99dc-72248084475b None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:49.181107 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdee2d-3dbe-4800-99dc-72248084475b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:49.203142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-4fcdee2d-3dbe-4800-99dc-72248084475b None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:02:49.204675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-4fcdee2d-3dbe-4800-99dc-72248084475b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:02:49.287734 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 604/1202] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:02:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:02:49.577232 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e990d-78dc-4471-92cf-4c9f769c1696 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:49.717050 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e990d-78dc-4471-92cf-4c9f769c1696 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:49.767032 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218af82c-56a3-4c0c-ae17-9c06db1e808a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:49.767679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218af82c-56a3-4c0c-ae17-9c06db1e808a None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:49.768070 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218af82c-56a3-4c0c-ae17-9c06db1e808a None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:49.768385 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218af82c-56a3-4c0c-ae17-9c06db1e808a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:49.781549 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218af82c-56a3-4c0c-ae17-9c06db1e808a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:49.783352 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218af82c-56a3-4c0c-ae17-9c06db1e808a None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:49.803150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-218af82c-56a3-4c0c-ae17-9c06db1e808a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:02:49.803150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:02:49.803150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:02:49.803150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:02:49.803150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:02:49.803150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:02:49.803150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:02:49.803150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:02:49.803150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:02:49.803150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:02:49.803150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:02:49.803150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:02:49.803150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:02:49.803150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:02:49.803150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:02:49.803150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:02:49.804338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:02:49.804338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:02:49.804338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:02:49.804338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:02:49.804338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:02:49.804338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:02:49.804338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:02:49.804338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:02:49.804338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:02:49.804338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:02:49.804338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:02:49.804338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:02:49.804338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:02:49.804338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:02:49.804338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:02:49.804338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:02:49.804338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:02:49.807133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:02:49.807133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:02:49.807133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:02:49.807133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 599/1203] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:02:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:02:49.810867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdee2d-3dbe-4800-99dc-72248084475b None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:49.936007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdee2d-3dbe-4800-99dc-72248084475b None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:49.969138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:02:49.977727 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34e09c-1640-4890-9dea-6c5a28cb5510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:49.978427 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34e09c-1640-4890-9dea-6c5a28cb5510 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:49.978711 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34e09c-1640-4890-9dea-6c5a28cb5510 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:49.979013 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34e09c-1640-4890-9dea-6c5a28cb5510 None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:49.979876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34e09c-1640-4890-9dea-6c5a28cb5510 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:02:49.980245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34e09c-1640-4890-9dea-6c5a28cb5510 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:49.981894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34e09c-1640-4890-9dea-6c5a28cb5510 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:49.998212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 605/1204] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:02:50.002891 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218af82c-56a3-4c0c-ae17-9c06db1e808a None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:50.074397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:02:50.145460 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218af82c-56a3-4c0c-ae17-9c06db1e808a None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:50.186258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287af1d6-6447-4570-b1c8-a749e8cb405d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:50.186912 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287af1d6-6447-4570-b1c8-a749e8cb405d None admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:50.187145 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287af1d6-6447-4570-b1c8-a749e8cb405d None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:50.187361 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287af1d6-6447-4570-b1c8-a749e8cb405d None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:50.188006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287af1d6-6447-4570-b1c8-a749e8cb405d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:50.189435 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287af1d6-6447-4570-b1c8-a749e8cb405d None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:50.225735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 600/1205] 10.222.0.73 () {60 vars in 1132 bytes} [Mon Aug 29 20:02:50 2022] POST /identity/v3/users => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:02:52.297126 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4afeac-ef49-452b-833b-51d8929a5e3c None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:52.297126 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4afeac-ef49-452b-833b-51d8929a5e3c None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:52.297126 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4afeac-ef49-452b-833b-51d8929a5e3c None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:52.297126 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 606/1206] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:02:52.304203 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db6d72-c180-4cfc-bcbf-674b628a3e02 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:52.304809 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db6d72-c180-4cfc-bcbf-674b628a3e02 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:52.304809 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db6d72-c180-4cfc-bcbf-674b628a3e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:52.328098 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-43db6d72-c180-4cfc-bcbf-674b628a3e02 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:02:52.329467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-43db6d72-c180-4cfc-bcbf-674b628a3e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:02:52.413871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 601/1207] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:02:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:02:52.689295 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4afeac-ef49-452b-833b-51d8929a5e3c None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:52.800189 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4afeac-ef49-452b-833b-51d8929a5e3c None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:52.844714 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fb55c-8e96-450d-bebf-b7b2767b4c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:52.845639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fb55c-8e96-450d-bebf-b7b2767b4c59 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:52.846182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fb55c-8e96-450d-bebf-b7b2767b4c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:52.846668 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fb55c-8e96-450d-bebf-b7b2767b4c59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:52.858760 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9fb55c-8e96-450d-bebf-b7b2767b4c59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:02:52.863749 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-7c9fb55c-8e96-450d-bebf-b7b2767b4c59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:02:52.863749 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:02:52.863749 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:02:52.863749 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:02:52.863749 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:02:52.863749 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:02:52.863749 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:02:52.863749 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:02:52.863749 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:02:52.863749 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:02:52.863749 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:02:52.863749 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:02:52.863749 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:02:52.863749 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:02:52.863749 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:02:52.863749 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:02:52.864859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:02:52.864859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:02:52.864859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:02:52.864859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:02:52.864859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:02:52.864859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:02:52.864859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:02:52.864859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:02:52.864859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:02:52.864859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:02:52.864859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:02:52.864859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:02:52.864859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:02:52.864859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:02:52.864859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:02:52.864859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:02:52.866421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:02:52.866421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:02:52.866421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:02:52.866421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:02:52.866421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:02:52.866421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:02:52.866421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:02:52.866421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:02:52.866421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:02:52.866421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:02:52.866421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:02:52.866421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:02:52.866421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 607/1208] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 20:02:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:02:52.870979 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db6d72-c180-4cfc-bcbf-674b628a3e02 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:52.914459 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:02:52.981225 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db6d72-c180-4cfc-bcbf-674b628a3e02 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:53.023741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093ffe0b-ad33-4c06-ac51-15d551baec79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:53.024642 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ffe0b-ad33-4c06-ac51-15d551baec79 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:53.024911 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ffe0b-ad33-4c06-ac51-15d551baec79 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:53.025178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ffe0b-ad33-4c06-ac51-15d551baec79 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:53.025899 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ffe0b-ad33-4c06-ac51-15d551baec79 None admin] RBAC: Adding query filter params (name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:02:53.026199 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ffe0b-ad33-4c06-ac51-15d551baec79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:53.027552 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ffe0b-ad33-4c06-ac51-15d551baec79 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:53.036411 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 602/1209] 10.222.0.73 () {56 vars in 1103 bytes} [Mon Aug 29 20:02:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:02:53.041776 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fb55c-8e96-450d-bebf-b7b2767b4c59 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:53.103635 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:02:53.169651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fb55c-8e96-450d-bebf-b7b2767b4c59 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:53.212610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314f0c9-e231-465a-85d8-d5e4a3e23bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:53.213244 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314f0c9-e231-465a-85d8-d5e4a3e23bda None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:53.213528 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314f0c9-e231-465a-85d8-d5e4a3e23bda None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:53.213833 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314f0c9-e231-465a-85d8-d5e4a3e23bda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:53.224628 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9314f0c9-e231-465a-85d8-d5e4a3e23bda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:53.228618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9314f0c9-e231-465a-85d8-d5e4a3e23bda None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:53.245928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-9314f0c9-e231-465a-85d8-d5e4a3e23bda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:02:53.245928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:02:53.245928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:02:53.245928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:02:53.245928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:02:53.245928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:02:53.245928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:02:53.245928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:02:53.245928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:02:53.245928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:02:53.245928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:02:53.245928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:02:53.245928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:02:53.245928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:02:53.245928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:02:53.245928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:02:53.247972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:02:53.247972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:02:53.247972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:02:53.247972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:02:53.247972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:02:53.247972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:02:53.247972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:02:53.247972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:02:53.247972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:02:53.247972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:02:53.247972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:02:53.247972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:02:53.247972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:02:53.247972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:02:53.247972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:02:53.247972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:02:53.247972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:02:53.249347 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:02:53.249347 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:02:53.249347 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:02:53.249347 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 608/1210] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:02:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:02:53.253190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093ffe0b-ad33-4c06-ac51-15d551baec79 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:53.370147 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:02:53.373990 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093ffe0b-ad33-4c06-ac51-15d551baec79 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:53.415639 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4d26e-adbe-44d4-9c2a-f96a7842def3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:53.416240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4d26e-adbe-44d4-9c2a-f96a7842def3 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:53.416521 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4d26e-adbe-44d4-9c2a-f96a7842def3 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:53.416812 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4d26e-adbe-44d4-9c2a-f96a7842def3 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:53.417501 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4d26e-adbe-44d4-9c2a-f96a7842def3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:02:53.417939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4d26e-adbe-44d4-9c2a-f96a7842def3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:53.419276 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4d26e-adbe-44d4-9c2a-f96a7842def3 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:53.435944 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 603/1211] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:02:53.441778 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314f0c9-e231-465a-85d8-d5e4a3e23bda None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:53.558192 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314f0c9-e231-465a-85d8-d5e4a3e23bda None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:53.559862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:02:53.597055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b3fbd-ac25-460f-ac31-771878f43787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:53.598264 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b3fbd-ac25-460f-ac31-771878f43787 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:53.598539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b3fbd-ac25-460f-ac31-771878f43787 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:53.598811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b3fbd-ac25-460f-ac31-771878f43787 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:53.606971 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b3fbd-ac25-460f-ac31-771878f43787 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:53.608410 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b3fbd-ac25-460f-ac31-771878f43787 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:53.621483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-ec1b3fbd-ac25-460f-ac31-771878f43787 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:02:53.621483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:02:53.621483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:02:53.621483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:02:53.621483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:02:53.621483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:02:53.621483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:02:53.621483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:02:53.621483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:02:53.621483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:02:53.621483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:02:53.621483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:02:53.621483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:02:53.621483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:02:53.621483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:02:53.621483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:02:53.622729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:02:53.622729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:02:53.622729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:02:53.622729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:02:53.622729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:02:53.622729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:02:53.622729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:02:53.622729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:02:53.622729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:02:53.622729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:02:53.622729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:02:53.622729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:02:53.622729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:02:53.622729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:02:53.622729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:02:53.622729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:02:53.622729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:02:53.622729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:02:53.623903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:02:53.623903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:02:53.623903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:02:53.623903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:02:53.623903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:02:53.623903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:02:53.623903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:02:53.623903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:02:53.623903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:02:53.623903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:02:53.627394 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 609/1212] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:02:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:02:53.629033 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4d26e-adbe-44d4-9c2a-f96a7842def3 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:53.743034 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4d26e-adbe-44d4-9c2a-f96a7842def3 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:53.784452 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be167e70-676b-4655-9153-f1db2b577182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:53.785920 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be167e70-676b-4655-9153-f1db2b577182 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:53.786213 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be167e70-676b-4655-9153-f1db2b577182 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:53.786525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be167e70-676b-4655-9153-f1db2b577182 None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:53.787203 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be167e70-676b-4655-9153-f1db2b577182 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:02:53.787502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be167e70-676b-4655-9153-f1db2b577182 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:53.789010 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be167e70-676b-4655-9153-f1db2b577182 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:53.807315 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 604/1213] 10.222.0.73 () {56 vars in 1143 bytes} [Mon Aug 29 20:02:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:02:53.813249 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b3fbd-ac25-460f-ac31-771878f43787 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:53.923766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b3fbd-ac25-460f-ac31-771878f43787 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:53.969144 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9fcad-bfd8-4658-821b-b1b14d5e0f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:53.969848 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9fcad-bfd8-4658-821b-b1b14d5e0f88 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:53.970111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9fcad-bfd8-4658-821b-b1b14d5e0f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:53.970384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9fcad-bfd8-4658-821b-b1b14d5e0f88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:54.002494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:02:54.045984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9fcad-bfd8-4658-821b-b1b14d5e0f88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:54.047442 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9fcad-bfd8-4658-821b-b1b14d5e0f88 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:54.172338 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 610/1214] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:02:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 20:02:54.179382 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be167e70-676b-4655-9153-f1db2b577182 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:54.201012 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:02:54.292179 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be167e70-676b-4655-9153-f1db2b577182 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:54.329021 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05472d4b-2758-4356-8cb3-e87f87a52cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:54.329636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05472d4b-2758-4356-8cb3-e87f87a52cdb None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:54.329959 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05472d4b-2758-4356-8cb3-e87f87a52cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:54.330233 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05472d4b-2758-4356-8cb3-e87f87a52cdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:54.341984 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05472d4b-2758-4356-8cb3-e87f87a52cdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:54.343396 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05472d4b-2758-4356-8cb3-e87f87a52cdb None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:54.361130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-05472d4b-2758-4356-8cb3-e87f87a52cdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:02:54.361130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:02:54.361130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:02:54.361130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:02:54.361130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:02:54.361130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:02:54.361130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:02:54.361130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:02:54.361130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:02:54.361130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:02:54.361130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:02:54.361130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:02:54.361130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:02:54.361130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:02:54.361130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:02:54.361130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:02:54.362375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:02:54.362375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:02:54.362375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:02:54.362375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:02:54.362375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:02:54.362375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:02:54.362375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:02:54.362375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:02:54.362375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:02:54.362375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:02:54.362375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:02:54.362375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:02:54.362375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:02:54.362375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:02:54.362375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:02:54.362375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:02:54.362375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:02:54.363562 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:02:54.363562 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:02:54.363562 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:02:54.367228 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 605/1215] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:02:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:02:54.368704 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9fcad-bfd8-4658-821b-b1b14d5e0f88 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:54.437829 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:02:54.482493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9fcad-bfd8-4658-821b-b1b14d5e0f88 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:54.520118 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7742e-2882-41f1-b41a-09a143fbf9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:54.520747 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7742e-2882-41f1-b41a-09a143fbf9bb None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:54.521018 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7742e-2882-41f1-b41a-09a143fbf9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:54.521291 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7742e-2882-41f1-b41a-09a143fbf9bb None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:54.521982 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7742e-2882-41f1-b41a-09a143fbf9bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:02:54.522370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7742e-2882-41f1-b41a-09a143fbf9bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:54.523777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7742e-2882-41f1-b41a-09a143fbf9bb None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:54.536541 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 611/1216] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:02:54.542247 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05472d4b-2758-4356-8cb3-e87f87a52cdb None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:54.621239 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:02:54.655330 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05472d4b-2758-4356-8cb3-e87f87a52cdb None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:54.692911 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7c718-d350-4940-a113-fce14ec66a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:54.693552 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7c718-d350-4940-a113-fce14ec66a93 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:54.693875 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7c718-d350-4940-a113-fce14ec66a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:54.694127 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7c718-d350-4940-a113-fce14ec66a93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:54.701492 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e7c718-d350-4940-a113-fce14ec66a93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:54.702909 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e7c718-d350-4940-a113-fce14ec66a93 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:54.720129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-21e7c718-d350-4940-a113-fce14ec66a93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:02:54.720129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:02:54.720129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:02:54.720129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:02:54.720129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:02:54.720129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:02:54.720129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:02:54.720129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:02:54.720129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:02:54.720129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:02:54.720129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:02:54.720129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:02:54.720129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:02:54.720129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:02:54.720129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:02:54.721134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:02:54.721134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:02:54.721134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:02:54.721134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:02:54.721134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:02:54.721134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:02:54.721134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:02:54.721134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:02:54.721134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:02:54.721134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:02:54.721134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:02:54.721134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:02:54.721134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:02:54.721134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:02:54.721134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:02:54.721134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:02:54.721134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:02:54.722132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:02:54.722132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:02:54.722132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:02:54.722132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:02:54.722132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:02:54.722132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:02:54.722132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:02:54.722132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:02:54.722132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 606/1217] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:02:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:02:54.727420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7742e-2882-41f1-b41a-09a143fbf9bb None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:54.854063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7742e-2882-41f1-b41a-09a143fbf9bb None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:54.880682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:02:54.893262 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a32349-bedf-427b-8165-3d3d84af52af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:54.893894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a32349-bedf-427b-8165-3d3d84af52af None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:54.894172 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a32349-bedf-427b-8165-3d3d84af52af None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:54.894524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a32349-bedf-427b-8165-3d3d84af52af None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:54.895546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a32349-bedf-427b-8165-3d3d84af52af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:02:54.895865 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a32349-bedf-427b-8165-3d3d84af52af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:54.897420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a32349-bedf-427b-8165-3d3d84af52af None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:54.911489 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 612/1218] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:02:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:02:54.917437 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7c718-d350-4940-a113-fce14ec66a93 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:55.031673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7c718-d350-4940-a113-fce14ec66a93 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:55.062830 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:02:55.070900 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b3c4e-5adb-4a0e-bc10-2fe20dc21753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:55.071818 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b3c4e-5adb-4a0e-bc10-2fe20dc21753 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:55.072141 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b3c4e-5adb-4a0e-bc10-2fe20dc21753 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:55.072432 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b3c4e-5adb-4a0e-bc10-2fe20dc21753 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:55.073294 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b3c4e-5adb-4a0e-bc10-2fe20dc21753 None admin] RBAC: Adding query filter params (role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=1ab2e318b2854c46bc7ed2151a44a2e2) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:02:55.073608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b3c4e-5adb-4a0e-bc10-2fe20dc21753 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:55.075500 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b3c4e-5adb-4a0e-bc10-2fe20dc21753 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:55.093732 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 607/1219] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:02:54 2022] GET /identity/v3/role_assignments?user.id=1ab2e318b2854c46bc7ed2151a44a2e2&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:02:57.094499 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229e01b-049d-43c4-bbba-730284058e7d None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:57.094992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229e01b-049d-43c4-bbba-730284058e7d None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:57.094992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229e01b-049d-43c4-bbba-730284058e7d None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:57.096364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 613/1220] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:02:57.102760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c0803-6e58-4e46-aa03-b9099f3e05b2 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:57.103218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c0803-6e58-4e46-aa03-b9099f3e05b2 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:57.103438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c0803-6e58-4e46-aa03-b9099f3e05b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:57.122917 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-8e7c0803-6e58-4e46-aa03-b9099f3e05b2 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:02:57.124120 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-8e7c0803-6e58-4e46-aa03-b9099f3e05b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:02:57.200322 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 608/1221] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:02:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:02:57.479082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229e01b-049d-43c4-bbba-730284058e7d None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:57.591545 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229e01b-049d-43c4-bbba-730284058e7d None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:57.628482 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54afbda-b3c5-47d1-a303-0bb8c5f18c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:57.629263 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54afbda-b3c5-47d1-a303-0bb8c5f18c6b None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:57.629677 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54afbda-b3c5-47d1-a303-0bb8c5f18c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:57.629986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54afbda-b3c5-47d1-a303-0bb8c5f18c6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:57.642578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54afbda-b3c5-47d1-a303-0bb8c5f18c6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:02:57.648223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-a54afbda-b3c5-47d1-a303-0bb8c5f18c6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:02:57.648223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:02:57.648223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:02:57.648223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:02:57.648223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:02:57.648223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:02:57.648223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:02:57.648223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:02:57.648223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:02:57.648223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:02:57.648223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:02:57.648223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:02:57.648223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:02:57.648223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:02:57.648223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:02:57.648223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:02:57.649337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:02:57.649337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:02:57.649337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:02:57.649337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:02:57.649337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:02:57.649337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:02:57.649337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:02:57.649337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:02:57.649337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:02:57.649337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:02:57.649337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:02:57.649337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:02:57.649337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:02:57.649337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:02:57.649337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:02:57.649337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:02:57.650284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:02:57.650284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:02:57.650284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:02:57.650284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:02:57.650284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:02:57.650284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:02:57.650284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:02:57.650284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:02:57.650284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:02:57.650284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:02:57.650284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:02:57.650284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:02:57.650284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 614/1222] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 20:02:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:02:57.655485 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c0803-6e58-4e46-aa03-b9099f3e05b2 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:57.772468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c0803-6e58-4e46-aa03-b9099f3e05b2 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:57.798524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:02:57.811362 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ab9d6-6653-46bd-89e2-8f0bf97c9659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:57.812007 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ab9d6-6653-46bd-89e2-8f0bf97c9659 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:57.812337 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ab9d6-6653-46bd-89e2-8f0bf97c9659 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:57.812677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ab9d6-6653-46bd-89e2-8f0bf97c9659 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:57.813384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ab9d6-6653-46bd-89e2-8f0bf97c9659 None admin] RBAC: Adding query filter params (name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:02:57.813744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ab9d6-6653-46bd-89e2-8f0bf97c9659 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:57.815142 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ab9d6-6653-46bd-89e2-8f0bf97c9659 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:57.824230 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 609/1223] 10.222.0.73 () {56 vars in 1103 bytes} [Mon Aug 29 20:02:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:02:57.830197 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54afbda-b3c5-47d1-a303-0bb8c5f18c6b None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:57.874324 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:02:57.932669 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54afbda-b3c5-47d1-a303-0bb8c5f18c6b None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:57.970721 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a20ac-fb32-4e68-be92-eb304e4b30ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:57.971344 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a20ac-fb32-4e68-be92-eb304e4b30ff None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:57.971660 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a20ac-fb32-4e68-be92-eb304e4b30ff None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:57.971933 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a20ac-fb32-4e68-be92-eb304e4b30ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:57.982782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099a20ac-fb32-4e68-be92-eb304e4b30ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:57.984319 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099a20ac-fb32-4e68-be92-eb304e4b30ff None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:57.999371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-099a20ac-fb32-4e68-be92-eb304e4b30ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:02:57.999371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:02:57.999371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:02:57.999371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:02:57.999371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:02:57.999371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:02:57.999371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:02:57.999371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:02:57.999371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:02:57.999371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:02:57.999371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:02:57.999371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:02:57.999371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:02:57.999371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:02:57.999371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:02:57.999371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:02:58.000504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:02:58.000504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:02:58.000504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:02:58.000504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:02:58.000504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:02:58.000504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:02:58.000504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:02:58.000504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:02:58.000504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:02:58.000504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:02:58.000504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:02:58.000504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:02:58.000504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:02:58.000504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:02:58.000504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:02:58.000504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:02:58.000504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:02:58.001424 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:02:58.001424 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:02:58.001424 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:02:58.001424 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 615/1224] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:02:58.007100 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ab9d6-6653-46bd-89e2-8f0bf97c9659 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:58.117910 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ab9d6-6653-46bd-89e2-8f0bf97c9659 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:58.156391 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4d8c7-32d5-49f0-89eb-f985f817e2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:58.157008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4d8c7-32d5-49f0-89eb-f985f817e2cf None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:58.157290 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4d8c7-32d5-49f0-89eb-f985f817e2cf None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:58.157565 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4d8c7-32d5-49f0-89eb-f985f817e2cf None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:58.158252 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4d8c7-32d5-49f0-89eb-f985f817e2cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:02:58.158559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4d8c7-32d5-49f0-89eb-f985f817e2cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:58.159924 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4d8c7-32d5-49f0-89eb-f985f817e2cf None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:58.172150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 610/1225] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:02:58.178553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a20ac-fb32-4e68-be92-eb304e4b30ff None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:58.274946 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:02:58.281916 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a20ac-fb32-4e68-be92-eb304e4b30ff None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:58.317204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9552f5-4b87-42b0-bdd2-afeab4ab1390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:58.317946 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9552f5-4b87-42b0-bdd2-afeab4ab1390 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:58.318220 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9552f5-4b87-42b0-bdd2-afeab4ab1390 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:58.318565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9552f5-4b87-42b0-bdd2-afeab4ab1390 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:58.326298 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9552f5-4b87-42b0-bdd2-afeab4ab1390 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:58.327775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9552f5-4b87-42b0-bdd2-afeab4ab1390 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:58.339151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-3f9552f5-4b87-42b0-bdd2-afeab4ab1390 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:02:58.339151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:02:58.339151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:02:58.339151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:02:58.339151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:02:58.339151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:02:58.339151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:02:58.339151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:02:58.339151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:02:58.339151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:02:58.339151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:02:58.339151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:02:58.339151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:02:58.339151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:02:58.339151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:02:58.339151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:02:58.340182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:02:58.340182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:02:58.340182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:02:58.340182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:02:58.340182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:02:58.340182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:02:58.340182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:02:58.340182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:02:58.340182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:02:58.340182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:02:58.340182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:02:58.340182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:02:58.340182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:02:58.340182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:02:58.340182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:02:58.340182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:02:58.340182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:02:58.340182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:02:58.341529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:02:58.341529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:02:58.341529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:02:58.341529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:02:58.341529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:02:58.341529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:02:58.341529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:02:58.341529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:02:58.341529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:02:58.341529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:02:58.341529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 616/1226] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:02:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:02:58.346348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4d8c7-32d5-49f0-89eb-f985f817e2cf None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:58.401310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:02:58.469993 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4d8c7-32d5-49f0-89eb-f985f817e2cf None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:58.509725 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fa981-300b-481e-a172-390b7872eff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:58.510492 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fa981-300b-481e-a172-390b7872eff7 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:58.510861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fa981-300b-481e-a172-390b7872eff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:58.511142 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fa981-300b-481e-a172-390b7872eff7 None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:58.512406 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832fa981-300b-481e-a172-390b7872eff7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:02:58.512699 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832fa981-300b-481e-a172-390b7872eff7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:58.514433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832fa981-300b-481e-a172-390b7872eff7 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:58.536023 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 611/1227] 10.222.0.73 () {56 vars in 1143 bytes} [Mon Aug 29 20:02:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:02:58.541916 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9552f5-4b87-42b0-bdd2-afeab4ab1390 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:58.658600 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9552f5-4b87-42b0-bdd2-afeab4ab1390 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:58.699331 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9458203-12d4-4d61-8ac0-11f191277361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:58.700110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9458203-12d4-4d61-8ac0-11f191277361 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:58.700392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9458203-12d4-4d61-8ac0-11f191277361 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:58.700661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9458203-12d4-4d61-8ac0-11f191277361 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:58.713730 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9458203-12d4-4d61-8ac0-11f191277361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:58.715428 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9458203-12d4-4d61-8ac0-11f191277361 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:58.732900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-f9458203-12d4-4d61-8ac0-11f191277361 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:02:58.732900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:02:58.732900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:02:58.732900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:02:58.732900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:02:58.732900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:02:58.732900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:02:58.732900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:02:58.732900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:02:58.732900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:02:58.732900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:02:58.732900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:02:58.732900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:02:58.732900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:02:58.732900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:02:58.732900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:02:58.734025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:02:58.734025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:02:58.734025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:02:58.734025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:02:58.734025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:02:58.734025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:02:58.734025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:02:58.734025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:02:58.734025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:02:58.734025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:02:58.734025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:02:58.734025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:02:58.734025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:02:58.734025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:02:58.734025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:02:58.734025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:02:58.734025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:02:58.734943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:02:58.734943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:02:58.734943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:02:58.736648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 617/1228] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:02:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:02:58.743273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fa981-300b-481e-a172-390b7872eff7 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:58.858609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fa981-300b-481e-a172-390b7872eff7 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:58.861360 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:02:58.898754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cacdc5-d51b-45bd-b267-dc62d2fa8d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:58.899354 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cacdc5-d51b-45bd-b267-dc62d2fa8d12 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:58.899627 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cacdc5-d51b-45bd-b267-dc62d2fa8d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:58.899891 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cacdc5-d51b-45bd-b267-dc62d2fa8d12 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:58.900519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cacdc5-d51b-45bd-b267-dc62d2fa8d12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:02:58.901180 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cacdc5-d51b-45bd-b267-dc62d2fa8d12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:58.902566 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cacdc5-d51b-45bd-b267-dc62d2fa8d12 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:58.915168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 612/1229] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:02:58.921087 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9458203-12d4-4d61-8ac0-11f191277361 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:58.931354 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:02:59.033083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9458203-12d4-4d61-8ac0-11f191277361 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:59.078626 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514e8ed-edf7-4e91-9f7d-acdc1f60e657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:59.079252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514e8ed-edf7-4e91-9f7d-acdc1f60e657 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:59.079524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514e8ed-edf7-4e91-9f7d-acdc1f60e657 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:59.079791 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514e8ed-edf7-4e91-9f7d-acdc1f60e657 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:59.090312 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514e8ed-edf7-4e91-9f7d-acdc1f60e657 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:59.091811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514e8ed-edf7-4e91-9f7d-acdc1f60e657 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:59.104966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-1514e8ed-edf7-4e91-9f7d-acdc1f60e657 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:02:59.104966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:02:59.104966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:02:59.104966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:02:59.104966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:02:59.104966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:02:59.104966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:02:59.104966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:02:59.104966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:02:59.104966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:02:59.104966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:02:59.104966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:02:59.104966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:02:59.104966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:02:59.104966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:02:59.106258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:02:59.106258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:02:59.106258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:02:59.106258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:02:59.106258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:02:59.106258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:02:59.106258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:02:59.106258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:02:59.106258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:02:59.106258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:02:59.106258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:02:59.106258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:02:59.106258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:02:59.106258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:02:59.106258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:02:59.106258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:02:59.106258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:02:59.107354 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:02:59.107354 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:02:59.107354 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:02:59.107354 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:02:59.107354 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:02:59.107354 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:02:59.107354 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:02:59.107354 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:02:59.107354 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 618/1230] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:02:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:02:59.112241 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cacdc5-d51b-45bd-b267-dc62d2fa8d12 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:59.220281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cacdc5-d51b-45bd-b267-dc62d2fa8d12 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:59.256487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5aac2d-6355-4ef9-a2c2-dcefea3c8992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:59.257102 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5aac2d-6355-4ef9-a2c2-dcefea3c8992 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:59.257502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5aac2d-6355-4ef9-a2c2-dcefea3c8992 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:59.257691 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5aac2d-6355-4ef9-a2c2-dcefea3c8992 None admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:59.258413 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5aac2d-6355-4ef9-a2c2-dcefea3c8992 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:02:59.258720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5aac2d-6355-4ef9-a2c2-dcefea3c8992 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:59.260145 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5aac2d-6355-4ef9-a2c2-dcefea3c8992 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:59.273561 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 613/1231] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:02:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:02:59.280003 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514e8ed-edf7-4e91-9f7d-acdc1f60e657 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:02:59.350114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:02:59.386479 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514e8ed-edf7-4e91-9f7d-acdc1f60e657 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:02:59.424200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3118843-401b-4fa7-a31f-6b008af79d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:02:59.424986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3118843-401b-4fa7-a31f-6b008af79d96 None admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:02:59.425270 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3118843-401b-4fa7-a31f-6b008af79d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:02:59.425547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3118843-401b-4fa7-a31f-6b008af79d96 None admin] PATH_INFO: `/v3/projects/2f5ec615d98548159487c97cead336a0/users/1ab2e318b2854c46bc7ed2151a44a2e2/roles/8ca2ec70f49a4f4da3ce9f3ac0b73083` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:02:59.443802 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3118843-401b-4fa7-a31f-6b008af79d96 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f5ec615d98548159487c97cead336a0, user_id=1ab2e318b2854c46bc7ed2151a44a2e2, role_id=8ca2ec70f49a4f4da3ce9f3ac0b73083)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:02:59.445253 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3118843-401b-4fa7-a31f-6b008af79d96 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:02:59.457964 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 619/1232] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 20:02:59 2022] PUT /identity/v3/projects/2f5ec615d98548159487c97cead336a0/users/1ab2e318b2854c46bc7ed2151a44a2e2/roles/8ca2ec70f49a4f4da3ce9f3ac0b73083 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:03:01.513079 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d77e49-279e-409c-8b13-bc235fb63ea1 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:01.513597 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d77e49-279e-409c-8b13-bc235fb63ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:01.513597 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d77e49-279e-409c-8b13-bc235fb63ea1 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:01.514922 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 614/1233] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:03:01.520317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a4051-489e-451a-8611-02220706139b None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:01.520757 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a4051-489e-451a-8611-02220706139b None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:01.520926 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a4051-489e-451a-8611-02220706139b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:01.541000 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-059a4051-489e-451a-8611-02220706139b None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:03:01.542248 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-059a4051-489e-451a-8611-02220706139b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:03:01.611989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 620/1234] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:03:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:03:01.895283 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d77e49-279e-409c-8b13-bc235fb63ea1 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:01.922767 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:02.008930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d77e49-279e-409c-8b13-bc235fb63ea1 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:02.049377 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcce06-214a-46cd-b84f-778feec3d665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:02.050082 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcce06-214a-46cd-b84f-778feec3d665 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:02.050442 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcce06-214a-46cd-b84f-778feec3d665 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:02.050758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcce06-214a-46cd-b84f-778feec3d665 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:02.062010 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbcce06-214a-46cd-b84f-778feec3d665 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:03:02.067563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-6bbcce06-214a-46cd-b84f-778feec3d665 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:03:02.067563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:02.067563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:03:02.067563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:03:02.067563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:03:02.067563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:03:02.067563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:03:02.067563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:03:02.067563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:03:02.067563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:03:02.067563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:03:02.067563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:03:02.067563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:03:02.067563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:03:02.067563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:03:02.067563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:03:02.068670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:03:02.068670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:03:02.068670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:03:02.068670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:03:02.068670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:03:02.068670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:03:02.068670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:03:02.068670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:03:02.068670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:03:02.068670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:03:02.068670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:03:02.068670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:03:02.068670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:03:02.068670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:03:02.068670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:03:02.068670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:03:02.069514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:03:02.069514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:03:02.069514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:03:02.069514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:03:02.069514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:03:02.069514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:03:02.069514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:03:02.069514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:03:02.069514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:03:02.069514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:03:02.069514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:03:02.069514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:03:02.069514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 615/1235] 10.222.0.73 () {56 vars in 1102 bytes} [Mon Aug 29 20:03:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:03:02.076013 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a4051-489e-451a-8611-02220706139b None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:02.126877 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:02.187689 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a4051-489e-451a-8611-02220706139b None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:02.226777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74446b19-e4c1-475c-8345-dc7a6e8f98dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:02.227384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74446b19-e4c1-475c-8345-dc7a6e8f98dc None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:02.227662 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74446b19-e4c1-475c-8345-dc7a6e8f98dc None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:02.227933 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74446b19-e4c1-475c-8345-dc7a6e8f98dc None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:02.228591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74446b19-e4c1-475c-8345-dc7a6e8f98dc None admin] RBAC: Adding query filter params (name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:02.228893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74446b19-e4c1-475c-8345-dc7a6e8f98dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:02.230293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74446b19-e4c1-475c-8345-dc7a6e8f98dc None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:02.238452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 621/1236] 10.222.0.73 () {56 vars in 1103 bytes} [Mon Aug 29 20:03:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:02.244980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcce06-214a-46cd-b84f-778feec3d665 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:02.335763 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:02.355157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcce06-214a-46cd-b84f-778feec3d665 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:02.395393 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af8193-fd30-4fd3-bc2a-e469d653e4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:02.396095 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af8193-fd30-4fd3-bc2a-e469d653e4bf None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:02.396384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af8193-fd30-4fd3-bc2a-e469d653e4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:02.396713 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af8193-fd30-4fd3-bc2a-e469d653e4bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:02.410482 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af8193-fd30-4fd3-bc2a-e469d653e4bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:02.412256 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af8193-fd30-4fd3-bc2a-e469d653e4bf None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:02.432592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-20af8193-fd30-4fd3-bc2a-e469d653e4bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:03:02.432592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:02.432592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:03:02.432592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:03:02.432592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:03:02.432592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:03:02.432592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:03:02.432592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:03:02.432592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:03:02.432592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:03:02.432592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:03:02.432592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:03:02.432592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:03:02.432592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:03:02.432592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:03:02.432592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:03:02.434072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:03:02.434072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:03:02.434072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:03:02.434072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:03:02.434072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:03:02.434072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:03:02.434072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:03:02.434072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:03:02.434072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:03:02.434072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:03:02.434072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:03:02.434072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:03:02.434072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:03:02.434072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:03:02.434072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:03:02.434072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:03:02.434072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:03:02.435236 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:03:02.435236 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:03:02.435236 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:03:02.435236 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 616/1237] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:03:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:03:02.440918 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74446b19-e4c1-475c-8345-dc7a6e8f98dc None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:02.559566 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74446b19-e4c1-475c-8345-dc7a6e8f98dc None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:02.570159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:02.600065 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801bfda-aac6-4a6d-8ba5-b16a7937dadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:02.600717 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801bfda-aac6-4a6d-8ba5-b16a7937dadb None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:02.601001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801bfda-aac6-4a6d-8ba5-b16a7937dadb None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:02.601449 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801bfda-aac6-4a6d-8ba5-b16a7937dadb None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:02.602140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801bfda-aac6-4a6d-8ba5-b16a7937dadb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:02.602507 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801bfda-aac6-4a6d-8ba5-b16a7937dadb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:02.604223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801bfda-aac6-4a6d-8ba5-b16a7937dadb None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:02.618370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 622/1238] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:02.624684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af8193-fd30-4fd3-bc2a-e469d653e4bf None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:02.740383 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af8193-fd30-4fd3-bc2a-e469d653e4bf None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:02.777587 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f3044-d57c-4684-a9e3-3f8bdcdc5e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:02.778400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f3044-d57c-4684-a9e3-3f8bdcdc5e57 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:02.778688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f3044-d57c-4684-a9e3-3f8bdcdc5e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:02.778966 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f3044-d57c-4684-a9e3-3f8bdcdc5e57 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:02.782207 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:02.788296 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f3044-d57c-4684-a9e3-3f8bdcdc5e57 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:02.789921 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f3044-d57c-4684-a9e3-3f8bdcdc5e57 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:02.801972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-d68f3044-d57c-4684-a9e3-3f8bdcdc5e57 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:03:02.801972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:02.801972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:03:02.801972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:03:02.801972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:03:02.801972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:03:02.801972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:03:02.801972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:03:02.801972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:03:02.801972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:03:02.801972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:03:02.801972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:03:02.801972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:03:02.801972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:03:02.801972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:03:02.801972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:03:02.803078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:03:02.803078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:03:02.803078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:03:02.803078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:03:02.803078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:03:02.803078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:03:02.803078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:03:02.803078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:03:02.803078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:03:02.803078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:03:02.803078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:03:02.803078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:03:02.803078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:03:02.803078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:03:02.803078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:03:02.803078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:03:02.803078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:03:02.803078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:03:02.809504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:03:02.809504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:03:02.809504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:03:02.809504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:03:02.809504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:03:02.809504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:03:02.809504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:03:02.809504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:03:02.809504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:03:02.809504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:03:02.809504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 617/1239] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:03:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:03:02.810341 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801bfda-aac6-4a6d-8ba5-b16a7937dadb None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:02.923897 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801bfda-aac6-4a6d-8ba5-b16a7937dadb None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:02.961883 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2451d-abd2-4260-b366-df85e291f6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:02.962564 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2451d-abd2-4260-b366-df85e291f6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:02.962851 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2451d-abd2-4260-b366-df85e291f6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:02.963139 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2451d-abd2-4260-b366-df85e291f6f0 None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:02.963932 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac2451d-abd2-4260-b366-df85e291f6f0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:02.964304 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac2451d-abd2-4260-b366-df85e291f6f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:02.965986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac2451d-abd2-4260-b366-df85e291f6f0 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:02.985807 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 623/1240] 10.222.0.73 () {56 vars in 1143 bytes} [Mon Aug 29 20:03:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:02.992565 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f3044-d57c-4684-a9e3-3f8bdcdc5e57 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:03.118637 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f3044-d57c-4684-a9e3-3f8bdcdc5e57 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:03.156971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456fe07-cc06-416c-9ff0-dae04592e2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:03.157643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456fe07-cc06-416c-9ff0-dae04592e2a0 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:03.158027 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456fe07-cc06-416c-9ff0-dae04592e2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:03.158334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456fe07-cc06-416c-9ff0-dae04592e2a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:03.224783 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:03.233312 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456fe07-cc06-416c-9ff0-dae04592e2a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:03.234997 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456fe07-cc06-416c-9ff0-dae04592e2a0 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:03.345200 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 618/1241] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:03:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:03:03.352260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2451d-abd2-4260-b366-df85e291f6f0 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:03.384279 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:03.479113 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2451d-abd2-4260-b366-df85e291f6f0 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:03.520532 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a79aef-362f-417f-a010-68d803c5a762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:03.521309 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a79aef-362f-417f-a010-68d803c5a762 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:03.521587 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a79aef-362f-417f-a010-68d803c5a762 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:03.521898 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a79aef-362f-417f-a010-68d803c5a762 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:03.534011 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a79aef-362f-417f-a010-68d803c5a762 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:03.536485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a79aef-362f-417f-a010-68d803c5a762 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:03.552574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-50a79aef-362f-417f-a010-68d803c5a762 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:03:03.552574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:03.552574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:03:03.552574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:03:03.552574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:03:03.552574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:03:03.552574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:03:03.552574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:03:03.552574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:03:03.552574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:03:03.552574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:03:03.552574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:03:03.552574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:03:03.552574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:03:03.552574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:03:03.552574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:03:03.554182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:03:03.554182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:03:03.554182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:03:03.554182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:03:03.554182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:03:03.554182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:03:03.554182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:03:03.554182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:03:03.554182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:03:03.554182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:03:03.554182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:03:03.554182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:03:03.554182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:03:03.554182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:03:03.554182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:03:03.554182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:03:03.554182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:03:03.555527 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:03:03.555527 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:03:03.555527 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:03:03.555527 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 624/1242] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:03:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:03:03.559652 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456fe07-cc06-416c-9ff0-dae04592e2a0 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:03.663233 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:03.670337 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456fe07-cc06-416c-9ff0-dae04592e2a0 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:03.709806 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d624ce5-b4cd-41ad-90c9-a093b128df0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:03.710459 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d624ce5-b4cd-41ad-90c9-a093b128df0e None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:03.710731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d624ce5-b4cd-41ad-90c9-a093b128df0e None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:03.711067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d624ce5-b4cd-41ad-90c9-a093b128df0e None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:03.711874 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d624ce5-b4cd-41ad-90c9-a093b128df0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:03.712089 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d624ce5-b4cd-41ad-90c9-a093b128df0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:03.713545 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d624ce5-b4cd-41ad-90c9-a093b128df0e None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:03.726234 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 619/1243] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:03.732106 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a79aef-362f-417f-a010-68d803c5a762 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:03.817999 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:03.841270 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a79aef-362f-417f-a010-68d803c5a762 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:03.879940 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351762ef-5528-4f22-9058-bb1f67e1724d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:03.880646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351762ef-5528-4f22-9058-bb1f67e1724d None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:03.880922 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351762ef-5528-4f22-9058-bb1f67e1724d None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:03.881197 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351762ef-5528-4f22-9058-bb1f67e1724d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:03.888690 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351762ef-5528-4f22-9058-bb1f67e1724d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:03.890401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351762ef-5528-4f22-9058-bb1f67e1724d None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:03.902559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-351762ef-5528-4f22-9058-bb1f67e1724d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:03:03.902559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:03.902559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:03:03.902559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:03:03.902559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:03:03.902559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:03:03.902559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:03:03.902559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:03:03.902559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:03:03.902559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:03:03.902559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:03:03.902559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:03:03.902559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:03:03.902559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:03:03.902559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:03:03.904063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:03:03.904063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:03:03.904063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:03:03.904063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:03:03.904063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:03:03.904063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:03:03.904063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:03:03.904063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:03:03.904063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:03:03.904063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:03:03.904063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:03:03.904063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:03:03.904063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:03:03.904063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:03:03.904063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:03:03.904063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:03:03.904063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:03:03.905001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:03:03.905001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:03:03.905001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:03:03.905001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:03:03.905001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:03:03.905001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:03:03.905001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:03:03.905001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:03:03.905001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 625/1244] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:03:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:03:03.909777 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d624ce5-b4cd-41ad-90c9-a093b128df0e None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:04.021019 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d624ce5-b4cd-41ad-90c9-a093b128df0e None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:04.058734 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed85020-ba59-42e2-b5e1-ee699a47d540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:04.059349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed85020-ba59-42e2-b5e1-ee699a47d540 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:04.059659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed85020-ba59-42e2-b5e1-ee699a47d540 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:04.059893 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed85020-ba59-42e2-b5e1-ee699a47d540 None admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:04.060576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed85020-ba59-42e2-b5e1-ee699a47d540 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:04.060881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed85020-ba59-42e2-b5e1-ee699a47d540 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:04.062371 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed85020-ba59-42e2-b5e1-ee699a47d540 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:04.075577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 620/1245] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:03:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:04.083245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351762ef-5528-4f22-9058-bb1f67e1724d None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:04.199651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351762ef-5528-4f22-9058-bb1f67e1724d None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:04.235538 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9198ab-7764-4774-b4f4-476bddb8bb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:04.236157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9198ab-7764-4774-b4f4-476bddb8bb11 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:04.236415 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9198ab-7764-4774-b4f4-476bddb8bb11 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:04.236681 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9198ab-7764-4774-b4f4-476bddb8bb11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:04.237428 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9198ab-7764-4774-b4f4-476bddb8bb11 None admin] RBAC: Adding query filter params (role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=1ab2e318b2854c46bc7ed2151a44a2e2) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:04.237754 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9198ab-7764-4774-b4f4-476bddb8bb11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:04.239798 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9198ab-7764-4774-b4f4-476bddb8bb11 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:04.247055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 626/1246] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:03:04 2022] GET /identity/v3/role_assignments?user.id=1ab2e318b2854c46bc7ed2151a44a2e2&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=8ca2ec70f49a4f4da3ce9f3ac0b73083 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:06.321429 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2194617-82d2-4cc0-8ccf-460707a7917d None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:06.322065 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2194617-82d2-4cc0-8ccf-460707a7917d None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:06.322065 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2194617-82d2-4cc0-8ccf-460707a7917d None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:06.323433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 621/1247] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:03:06.328935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a11e18-ad2d-42b0-9f33-d078aba870bc None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:06.329353 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a11e18-ad2d-42b0-9f33-d078aba870bc None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:06.329512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a11e18-ad2d-42b0-9f33-d078aba870bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:06.336501 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:06.353445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-b0a11e18-ad2d-42b0-9f33-d078aba870bc None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:03:06.354939 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-b0a11e18-ad2d-42b0-9f33-d078aba870bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:03:06.438490 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 627/1248] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:03:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:03:06.716034 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2194617-82d2-4cc0-8ccf-460707a7917d None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:06.737269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:06.832157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2194617-82d2-4cc0-8ccf-460707a7917d None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:06.869750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d0d11-9627-4f14-8aef-ab8724393122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:06.870714 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d0d11-9627-4f14-8aef-ab8724393122 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:06.871155 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d0d11-9627-4f14-8aef-ab8724393122 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:06.871474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d0d11-9627-4f14-8aef-ab8724393122 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:06.882984 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d0d11-9627-4f14-8aef-ab8724393122 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:03:06.888523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-249d0d11-9627-4f14-8aef-ab8724393122 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:03:06.888523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:06.888523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:03:06.888523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:03:06.888523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:03:06.888523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:03:06.888523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:03:06.888523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:03:06.888523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:03:06.888523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:03:06.888523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:03:06.888523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:03:06.888523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:03:06.888523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:03:06.888523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:03:06.888523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:03:06.893761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:03:06.893761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:03:06.893761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:03:06.893761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:03:06.893761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:03:06.893761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:03:06.893761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:03:06.893761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:03:06.893761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:03:06.893761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:03:06.893761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:03:06.893761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:03:06.893761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:03:06.893761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:03:06.893761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:03:06.893761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:03:06.894913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:03:06.894913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:03:06.894913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:03:06.894913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:03:06.894913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:03:06.894913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:03:06.894913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:03:06.894913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:03:06.894913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:03:06.894913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:03:06.894913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:03:06.894913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:03:06.894913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 622/1249] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 20:03:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:03:06.896339 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a11e18-ad2d-42b0-9f33-d078aba870bc None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:07.008663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a11e18-ad2d-42b0-9f33-d078aba870bc None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:07.045941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:07.053098 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719d1b0-1ff1-4dab-8eaa-28efa0e45062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:07.053810 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719d1b0-1ff1-4dab-8eaa-28efa0e45062 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:07.054090 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719d1b0-1ff1-4dab-8eaa-28efa0e45062 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:07.054366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719d1b0-1ff1-4dab-8eaa-28efa0e45062 None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:07.055141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4719d1b0-1ff1-4dab-8eaa-28efa0e45062 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:07.055481 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4719d1b0-1ff1-4dab-8eaa-28efa0e45062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:07.057574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4719d1b0-1ff1-4dab-8eaa-28efa0e45062 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:07.067200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 628/1250] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 20:03:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:07.073201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d0d11-9627-4f14-8aef-ab8724393122 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:07.167564 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:07.192844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d0d11-9627-4f14-8aef-ab8724393122 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:07.239784 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1f5b4-3c8f-4303-86be-c88e731703fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:07.240596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1f5b4-3c8f-4303-86be-c88e731703fe None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:07.240921 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1f5b4-3c8f-4303-86be-c88e731703fe None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:07.241273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1f5b4-3c8f-4303-86be-c88e731703fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:07.253064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1f5b4-3c8f-4303-86be-c88e731703fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:07.255477 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1f5b4-3c8f-4303-86be-c88e731703fe None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:07.272933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-f2e1f5b4-3c8f-4303-86be-c88e731703fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:03:07.272933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:07.272933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:03:07.272933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:03:07.272933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:03:07.272933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:03:07.272933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:03:07.272933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:03:07.272933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:03:07.272933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:03:07.272933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:03:07.272933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:03:07.272933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:03:07.272933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:03:07.272933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:03:07.272933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:03:07.274288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:03:07.274288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:03:07.274288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:03:07.274288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:03:07.274288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:03:07.274288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:03:07.274288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:03:07.274288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:03:07.274288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:03:07.274288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:03:07.274288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:03:07.274288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:03:07.274288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:03:07.274288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:03:07.274288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:03:07.274288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:03:07.274288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:03:07.275347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:03:07.275347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:03:07.275347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:03:07.279572 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 623/1251] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:03:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:03:07.280426 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719d1b0-1ff1-4dab-8eaa-28efa0e45062 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:07.398867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719d1b0-1ff1-4dab-8eaa-28efa0e45062 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:07.445898 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0152b-b431-4914-a5d6-9c623a2bb132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:07.446973 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0152b-b431-4914-a5d6-9c623a2bb132 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:07.447348 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0152b-b431-4914-a5d6-9c623a2bb132 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:07.447643 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0152b-b431-4914-a5d6-9c623a2bb132 None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:07.448490 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b0152b-b431-4914-a5d6-9c623a2bb132 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:07.448851 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b0152b-b431-4914-a5d6-9c623a2bb132 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:07.450359 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b0152b-b431-4914-a5d6-9c623a2bb132 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:07.464655 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 629/1252] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:03:07.470665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1f5b4-3c8f-4303-86be-c88e731703fe None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:07.583227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1f5b4-3c8f-4303-86be-c88e731703fe None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:07.609932 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:07.618159 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676554ed-8764-4301-bbc7-8474879c1523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:07.618778 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676554ed-8764-4301-bbc7-8474879c1523 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:07.619068 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676554ed-8764-4301-bbc7-8474879c1523 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:07.619339 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676554ed-8764-4301-bbc7-8474879c1523 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:07.627347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676554ed-8764-4301-bbc7-8474879c1523 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:07.628800 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676554ed-8764-4301-bbc7-8474879c1523 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:07.641211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-676554ed-8764-4301-bbc7-8474879c1523 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:03:07.641211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:07.641211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:03:07.641211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:03:07.641211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:03:07.641211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:03:07.641211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:03:07.641211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:03:07.641211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:03:07.641211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:03:07.641211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:03:07.641211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:03:07.641211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:03:07.641211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:03:07.641211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:03:07.641211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:03:07.642740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:03:07.642740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:03:07.642740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:03:07.642740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:03:07.642740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:03:07.642740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:03:07.642740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:03:07.642740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:03:07.642740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:03:07.642740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:03:07.642740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:03:07.642740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:03:07.642740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:03:07.642740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:03:07.642740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:03:07.642740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:03:07.642740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:03:07.642740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:03:07.643893 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:03:07.643893 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:03:07.643893 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:03:07.643893 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:03:07.643893 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:03:07.643893 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:03:07.643893 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:03:07.643893 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:03:07.643893 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:03:07.643893 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:03:07.643893 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 624/1253] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:03:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:03:07.649165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0152b-b431-4914-a5d6-9c623a2bb132 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:07.714902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:07.763644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0152b-b431-4914-a5d6-9c623a2bb132 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:07.797157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b92c3-c06e-4b84-bee4-d63904696f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:07.797783 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b92c3-c06e-4b84-bee4-d63904696f46 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:07.798061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b92c3-c06e-4b84-bee4-d63904696f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:07.798329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b92c3-c06e-4b84-bee4-d63904696f46 None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:07.799040 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b92c3-c06e-4b84-bee4-d63904696f46 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:07.799340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b92c3-c06e-4b84-bee4-d63904696f46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:07.800762 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b92c3-c06e-4b84-bee4-d63904696f46 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:07.818731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 630/1254] 10.222.0.73 () {56 vars in 1143 bytes} [Mon Aug 29 20:03:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:07.824563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676554ed-8764-4301-bbc7-8474879c1523 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:07.941615 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676554ed-8764-4301-bbc7-8474879c1523 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:07.979871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef912000-c277-441b-a0d7-6843defca477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:07.980565 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef912000-c277-441b-a0d7-6843defca477 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:07.980894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef912000-c277-441b-a0d7-6843defca477 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:07.981125 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef912000-c277-441b-a0d7-6843defca477 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:08.040882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:08.054866 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef912000-c277-441b-a0d7-6843defca477 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:08.056316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef912000-c277-441b-a0d7-6843defca477 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:08.163207 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 625/1255] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:03:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:03:08.168988 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b92c3-c06e-4b84-bee4-d63904696f46 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:08.276647 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b92c3-c06e-4b84-bee4-d63904696f46 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:08.290212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:08.312827 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7706e95-cba1-43d9-aada-94b9971678d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:08.313484 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7706e95-cba1-43d9-aada-94b9971678d1 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:08.313886 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7706e95-cba1-43d9-aada-94b9971678d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:08.314246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7706e95-cba1-43d9-aada-94b9971678d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:08.325866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7706e95-cba1-43d9-aada-94b9971678d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:08.327527 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7706e95-cba1-43d9-aada-94b9971678d1 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:08.344716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-e7706e95-cba1-43d9-aada-94b9971678d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:03:08.344716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:08.344716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:03:08.344716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:03:08.344716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:03:08.344716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:03:08.344716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:03:08.344716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:03:08.344716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:03:08.344716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:03:08.344716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:03:08.344716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:03:08.344716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:03:08.344716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:03:08.344716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:03:08.344716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:03:08.345867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:03:08.345867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:03:08.345867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:03:08.345867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:03:08.345867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:03:08.345867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:03:08.345867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:03:08.345867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:03:08.345867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:03:08.345867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:03:08.345867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:03:08.345867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:03:08.345867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:03:08.345867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:03:08.345867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:03:08.345867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:03:08.345867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:03:08.346900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:03:08.346900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:03:08.346900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:03:08.346900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 631/1256] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:03:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:03:08.352770 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef912000-c277-441b-a0d7-6843defca477 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:08.459830 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:08.476735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef912000-c277-441b-a0d7-6843defca477 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:08.510892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0561c6d-44fa-4608-91b3-472553ccd635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:08.511496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0561c6d-44fa-4608-91b3-472553ccd635 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:08.511776 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0561c6d-44fa-4608-91b3-472553ccd635 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:08.512049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0561c6d-44fa-4608-91b3-472553ccd635 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:08.514188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0561c6d-44fa-4608-91b3-472553ccd635 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:08.514188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0561c6d-44fa-4608-91b3-472553ccd635 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:08.514621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0561c6d-44fa-4608-91b3-472553ccd635 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:08.526813 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 626/1257] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:03:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:08.532212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7706e95-cba1-43d9-aada-94b9971678d1 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:08.641257 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7706e95-cba1-43d9-aada-94b9971678d1 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:08.676960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670159b-6627-4e1d-a992-00f8d1147edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:08.677581 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670159b-6627-4e1d-a992-00f8d1147edd None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:08.677882 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670159b-6627-4e1d-a992-00f8d1147edd None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:08.678149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670159b-6627-4e1d-a992-00f8d1147edd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:08.684885 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670159b-6627-4e1d-a992-00f8d1147edd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:08.686358 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670159b-6627-4e1d-a992-00f8d1147edd None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:08.698700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-6670159b-6627-4e1d-a992-00f8d1147edd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:03:08.698700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:08.698700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:03:08.698700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:03:08.698700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:03:08.698700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:03:08.698700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:03:08.698700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:03:08.698700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:03:08.698700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:03:08.698700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:03:08.698700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:03:08.698700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:03:08.698700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:03:08.698700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:03:08.700022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:03:08.700022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:03:08.700022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:03:08.700022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:03:08.700022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:03:08.700022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:03:08.700022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:03:08.700022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:03:08.700022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:03:08.700022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:03:08.700022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:03:08.700022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:03:08.700022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:03:08.700022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:03:08.700022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:03:08.700022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:03:08.700022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:03:08.701088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:03:08.701088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:03:08.701088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:03:08.701088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:03:08.701088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:03:08.701088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:03:08.701088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:03:08.701088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:03:08.701088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 632/1258] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:03:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:03:08.706263 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0561c6d-44fa-4608-91b3-472553ccd635 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:08.814045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0561c6d-44fa-4608-91b3-472553ccd635 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:08.851752 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a17b6-2689-4ebf-b864-0ce563b468f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:08.852356 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a17b6-2689-4ebf-b864-0ce563b468f2 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:08.852636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a17b6-2689-4ebf-b864-0ce563b468f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:08.852907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a17b6-2689-4ebf-b864-0ce563b468f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:08.853592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a17b6-2689-4ebf-b864-0ce563b468f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:08.853926 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a17b6-2689-4ebf-b864-0ce563b468f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:08.855350 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a17b6-2689-4ebf-b864-0ce563b468f2 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:08.868313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 627/1259] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:03:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:08.874464 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670159b-6627-4e1d-a992-00f8d1147edd None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:08.901680 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:08.983265 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670159b-6627-4e1d-a992-00f8d1147edd None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:09.024040 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ef245-2dbf-4ad8-a214-3415bd45c8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:09.024650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ef245-2dbf-4ad8-a214-3415bd45c8a8 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:09.024932 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ef245-2dbf-4ad8-a214-3415bd45c8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:09.025203 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ef245-2dbf-4ad8-a214-3415bd45c8a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:09.025979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ef245-2dbf-4ad8-a214-3415bd45c8a8 None admin] RBAC: Adding query filter params (role.id=296ee2c194824485af71618619f82e09, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=1ab2e318b2854c46bc7ed2151a44a2e2) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:09.026287 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ef245-2dbf-4ad8-a214-3415bd45c8a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:09.027795 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ef245-2dbf-4ad8-a214-3415bd45c8a8 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:09.033525 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 633/1260] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:03:08 2022] GET /identity/v3/role_assignments?user.id=1ab2e318b2854c46bc7ed2151a44a2e2&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=296ee2c194824485af71618619f82e09 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:10.936421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4073a1-465a-492e-94c3-f64b9bb30b09 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:10.936946 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4073a1-465a-492e-94c3-f64b9bb30b09 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:10.936946 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4073a1-465a-492e-94c3-f64b9bb30b09 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:10.938366 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 628/1261] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:03:10.944340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6049cb-c961-4a52-b7dd-07e5fa6e81b8 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:10.944853 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6049cb-c961-4a52-b7dd-07e5fa6e81b8 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:10.944946 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6049cb-c961-4a52-b7dd-07e5fa6e81b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:10.962988 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-fa6049cb-c961-4a52-b7dd-07e5fa6e81b8 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:03:10.964173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-fa6049cb-c961-4a52-b7dd-07e5fa6e81b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:03:11.030429 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 634/1262] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:03:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:03:11.298054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4073a1-465a-492e-94c3-f64b9bb30b09 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:11.309468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:11.398936 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4073a1-465a-492e-94c3-f64b9bb30b09 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:11.441227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28765d95-4b0f-4c22-8ee7-7b25bc07a323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:11.442322 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28765d95-4b0f-4c22-8ee7-7b25bc07a323 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:11.442791 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28765d95-4b0f-4c22-8ee7-7b25bc07a323 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:11.443206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28765d95-4b0f-4c22-8ee7-7b25bc07a323 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:11.455556 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28765d95-4b0f-4c22-8ee7-7b25bc07a323 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:03:11.461655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-28765d95-4b0f-4c22-8ee7-7b25bc07a323 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:03:11.461655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:11.461655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:03:11.461655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:03:11.461655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:03:11.461655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:03:11.461655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:03:11.461655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:03:11.461655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:03:11.461655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:03:11.461655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:03:11.461655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:03:11.461655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:03:11.461655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:03:11.461655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:03:11.461655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:03:11.462735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:03:11.462735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:03:11.462735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:03:11.462735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:03:11.462735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:03:11.462735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:03:11.462735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:03:11.462735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:03:11.462735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:03:11.462735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:03:11.462735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:03:11.462735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:03:11.462735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:03:11.462735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:03:11.462735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:03:11.462735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:03:11.463736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:03:11.463736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:03:11.463736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:03:11.463736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:03:11.463736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:03:11.463736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:03:11.463736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:03:11.463736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:03:11.463736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:03:11.463736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:03:11.463736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:03:11.463736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:03:11.463736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 629/1263] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 20:03:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:03:11.470332 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6049cb-c961-4a52-b7dd-07e5fa6e81b8 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:11.488854 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:11.567772 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6049cb-c961-4a52-b7dd-07e5fa6e81b8 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:11.600374 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6278d-f1e1-4c94-9d7f-75d1061ef01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:11.600958 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6278d-f1e1-4c94-9d7f-75d1061ef01c None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:11.601235 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6278d-f1e1-4c94-9d7f-75d1061ef01c None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:11.601503 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6278d-f1e1-4c94-9d7f-75d1061ef01c None admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:11.602190 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d6278d-f1e1-4c94-9d7f-75d1061ef01c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:11.602493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d6278d-f1e1-4c94-9d7f-75d1061ef01c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:11.603846 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d6278d-f1e1-4c94-9d7f-75d1061ef01c None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:11.611579 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 635/1264] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 20:03:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:03:11.618231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28765d95-4b0f-4c22-8ee7-7b25bc07a323 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:11.683675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:11.716929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28765d95-4b0f-4c22-8ee7-7b25bc07a323 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:11.749265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1b01c-f67b-453b-9f93-0d0619dfeb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:11.749911 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1b01c-f67b-453b-9f93-0d0619dfeb9f None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:11.750185 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1b01c-f67b-453b-9f93-0d0619dfeb9f None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:11.750451 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1b01c-f67b-453b-9f93-0d0619dfeb9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:11.760532 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1b01c-f67b-453b-9f93-0d0619dfeb9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:11.761980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1b01c-f67b-453b-9f93-0d0619dfeb9f None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:11.776512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-aea1b01c-f67b-453b-9f93-0d0619dfeb9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:03:11.776512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:11.776512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:03:11.776512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:03:11.776512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:03:11.776512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:03:11.776512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:03:11.776512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:03:11.776512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:03:11.776512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:03:11.776512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:03:11.776512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:03:11.776512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:03:11.776512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:03:11.776512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:03:11.776512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:03:11.781379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:03:11.781379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:03:11.781379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:03:11.781379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:03:11.781379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:03:11.781379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:03:11.781379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:03:11.781379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:03:11.781379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:03:11.781379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:03:11.781379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:03:11.781379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:03:11.781379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:03:11.781379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:03:11.781379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:03:11.781379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:03:11.781379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:03:11.782533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:03:11.782533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:03:11.782533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:03:11.782533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 630/1265] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:03:11.783496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6278d-f1e1-4c94-9d7f-75d1061ef01c None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:11.864857 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:11.884336 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6278d-f1e1-4c94-9d7f-75d1061ef01c None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:11.918576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ee13c-1ce9-4be0-b943-fde12dd1a70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:11.919169 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ee13c-1ce9-4be0-b943-fde12dd1a70e None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:11.919444 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ee13c-1ce9-4be0-b943-fde12dd1a70e None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:11.919712 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ee13c-1ce9-4be0-b943-fde12dd1a70e None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:11.920340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207ee13c-1ce9-4be0-b943-fde12dd1a70e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:11.920638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207ee13c-1ce9-4be0-b943-fde12dd1a70e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:11.922021 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207ee13c-1ce9-4be0-b943-fde12dd1a70e None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:11.933559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 636/1266] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:11.939346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1b01c-f67b-453b-9f93-0d0619dfeb9f None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:12.038621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1b01c-f67b-453b-9f93-0d0619dfeb9f None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:12.057556 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:12.072614 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb031a-e150-4b7c-9e1a-6f7d1fe36e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:12.073236 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb031a-e150-4b7c-9e1a-6f7d1fe36e89 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:12.073518 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb031a-e150-4b7c-9e1a-6f7d1fe36e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:12.073822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb031a-e150-4b7c-9e1a-6f7d1fe36e89 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:12.081060 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb031a-e150-4b7c-9e1a-6f7d1fe36e89 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:12.082627 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb031a-e150-4b7c-9e1a-6f7d1fe36e89 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:12.094471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-32eb031a-e150-4b7c-9e1a-6f7d1fe36e89 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:03:12.094471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:12.094471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:03:12.094471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:03:12.094471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:03:12.094471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:03:12.094471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:03:12.094471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:03:12.094471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:03:12.094471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:03:12.094471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:03:12.094471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:03:12.094471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:03:12.094471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:03:12.094471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:03:12.094471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:03:12.095487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:03:12.095487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:03:12.095487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:03:12.095487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:03:12.095487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:03:12.095487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:03:12.095487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:03:12.095487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:03:12.095487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:03:12.095487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:03:12.095487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:03:12.095487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:03:12.095487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:03:12.095487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:03:12.095487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:03:12.095487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:03:12.095487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:03:12.095487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:03:12.096438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:03:12.096438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:03:12.096438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:03:12.096438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:03:12.096438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:03:12.096438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:03:12.096438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:03:12.096438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:03:12.096438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:03:12.096438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:03:12.096438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 631/1267] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:03:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:03:12.101388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ee13c-1ce9-4be0-b943-fde12dd1a70e None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:12.197510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ee13c-1ce9-4be0-b943-fde12dd1a70e None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:12.231157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6f116-365a-4738-8995-7d5c7b379428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:12.231749 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6f116-365a-4738-8995-7d5c7b379428 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:12.232027 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6f116-365a-4738-8995-7d5c7b379428 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:12.232295 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6f116-365a-4738-8995-7d5c7b379428 None admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:12.233009 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6f116-365a-4738-8995-7d5c7b379428 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:12.233312 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6f116-365a-4738-8995-7d5c7b379428 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:12.234763 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6f116-365a-4738-8995-7d5c7b379428 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:12.243194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:12.251445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 637/1268] 10.222.0.73 () {56 vars in 1143 bytes} [Mon Aug 29 20:03:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:12.257659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb031a-e150-4b7c-9e1a-6f7d1fe36e89 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:12.355401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb031a-e150-4b7c-9e1a-6f7d1fe36e89 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:12.388005 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d99a2ae-6eb2-4921-8016-e0a49e6c59d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:12.388619 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99a2ae-6eb2-4921-8016-e0a49e6c59d6 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:12.388899 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99a2ae-6eb2-4921-8016-e0a49e6c59d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:12.389172 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99a2ae-6eb2-4921-8016-e0a49e6c59d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:12.400911 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d99a2ae-6eb2-4921-8016-e0a49e6c59d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:12.402652 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d99a2ae-6eb2-4921-8016-e0a49e6c59d6 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:12.417939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-9d99a2ae-6eb2-4921-8016-e0a49e6c59d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:03:12.417939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:12.417939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:03:12.417939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:03:12.417939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:03:12.417939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:03:12.417939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:03:12.417939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:03:12.417939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:03:12.417939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:03:12.417939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:03:12.417939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:03:12.417939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:03:12.417939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:03:12.417939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:03:12.417939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:03:12.419458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:03:12.419458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:03:12.419458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:03:12.419458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:03:12.419458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:03:12.419458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:03:12.419458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:03:12.419458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:03:12.419458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:03:12.419458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:03:12.419458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:03:12.419458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:03:12.419458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:03:12.419458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:03:12.419458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:03:12.419458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:03:12.419458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:03:12.420567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:03:12.420567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:03:12.420567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:03:12.420567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 632/1269] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:03:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:03:12.425025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6f116-365a-4738-8995-7d5c7b379428 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:12.523948 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6f116-365a-4738-8995-7d5c7b379428 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:12.557109 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22d667-f8f9-489a-bbbb-e52a35fe2aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:12.557699 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22d667-f8f9-489a-bbbb-e52a35fe2aec None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:12.558004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22d667-f8f9-489a-bbbb-e52a35fe2aec None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:12.558275 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22d667-f8f9-489a-bbbb-e52a35fe2aec None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:12.558904 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22d667-f8f9-489a-bbbb-e52a35fe2aec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:12.559204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22d667-f8f9-489a-bbbb-e52a35fe2aec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:12.560567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22d667-f8f9-489a-bbbb-e52a35fe2aec None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:12.572150 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 638/1270] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:12.577957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d99a2ae-6eb2-4921-8016-e0a49e6c59d6 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:12.608871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:12.676200 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d99a2ae-6eb2-4921-8016-e0a49e6c59d6 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:12.709404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af51957-8fa4-4a24-91fa-d5c301233345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:12.710041 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af51957-8fa4-4a24-91fa-d5c301233345 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:12.710395 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af51957-8fa4-4a24-91fa-d5c301233345 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:12.710666 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af51957-8fa4-4a24-91fa-d5c301233345 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:12.717051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af51957-8fa4-4a24-91fa-d5c301233345 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:12.718489 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af51957-8fa4-4a24-91fa-d5c301233345 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:12.729591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-0af51957-8fa4-4a24-91fa-d5c301233345 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:03:12.729591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:12.729591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:03:12.729591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:03:12.729591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:03:12.729591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:03:12.729591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:03:12.729591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:03:12.729591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:03:12.729591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:03:12.729591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:03:12.729591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:03:12.729591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:03:12.729591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:03:12.729591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:03:12.730674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:03:12.730674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:03:12.730674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:03:12.730674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:03:12.730674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:03:12.730674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:03:12.730674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:03:12.730674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:03:12.730674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:03:12.730674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:03:12.730674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:03:12.730674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:03:12.730674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:03:12.730674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:03:12.730674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:03:12.730674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:03:12.730674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:03:12.731611 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:03:12.731611 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:03:12.731611 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:03:12.731611 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:03:12.731611 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:03:12.731611 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:03:12.731611 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:03:12.731611 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:03:12.731611 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 633/1271] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:03:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:03:12.736652 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22d667-f8f9-489a-bbbb-e52a35fe2aec None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:12.794645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:12.835478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22d667-f8f9-489a-bbbb-e52a35fe2aec None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:12.868520 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053134d3-ce77-4cfc-80ef-755fafa32cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:12.869114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053134d3-ce77-4cfc-80ef-755fafa32cda None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:12.869394 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053134d3-ce77-4cfc-80ef-755fafa32cda None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:12.869665 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053134d3-ce77-4cfc-80ef-755fafa32cda None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:12.870376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053134d3-ce77-4cfc-80ef-755fafa32cda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:12.870681 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053134d3-ce77-4cfc-80ef-755fafa32cda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:12.872107 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053134d3-ce77-4cfc-80ef-755fafa32cda None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:12.883776 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 639/1272] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:03:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:12.890082 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af51957-8fa4-4a24-91fa-d5c301233345 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:12.993037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:13.002293 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af51957-8fa4-4a24-91fa-d5c301233345 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:13.034850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3aa847-dc23-44d6-9be2-c2d7b6129464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:13.035500 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aa847-dc23-44d6-9be2-c2d7b6129464 None admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:13.036316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aa847-dc23-44d6-9be2-c2d7b6129464 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:13.036673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aa847-dc23-44d6-9be2-c2d7b6129464 None admin] PATH_INFO: `/v3/projects/2f5ec615d98548159487c97cead336a0/users/1ab2e318b2854c46bc7ed2151a44a2e2/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:13.038604 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3aa847-dc23-44d6-9be2-c2d7b6129464 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f5ec615d98548159487c97cead336a0, user_id=1ab2e318b2854c46bc7ed2151a44a2e2, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:13.040014 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3aa847-dc23-44d6-9be2-c2d7b6129464 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:13.053554 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 634/1273] 10.222.0.73 () {58 vars in 1439 bytes} [Mon Aug 29 20:03:12 2022] PUT /identity/v3/projects/2f5ec615d98548159487c97cead336a0/users/1ab2e318b2854c46bc7ed2151a44a2e2/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:03:14.946840 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62115c-a72c-4b4e-bdf9-79aaaf2abbc4 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:14.947368 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62115c-a72c-4b4e-bdf9-79aaaf2abbc4 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:14.947368 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62115c-a72c-4b4e-bdf9-79aaaf2abbc4 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:14.948717 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 640/1274] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:03:14.953733 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be030c-6306-41bd-877c-3d7252b654ed None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:14.954167 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be030c-6306-41bd-877c-3d7252b654ed None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:14.954302 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be030c-6306-41bd-877c-3d7252b654ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:14.972542 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-06be030c-6306-41bd-877c-3d7252b654ed None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:03:14.973815 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-06be030c-6306-41bd-877c-3d7252b654ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:03:15.040390 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 635/1275] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:03:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:03:15.315892 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62115c-a72c-4b4e-bdf9-79aaaf2abbc4 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:15.420746 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62115c-a72c-4b4e-bdf9-79aaaf2abbc4 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:15.440096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:15.456488 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd838b98-49a7-4590-99e4-1442de35a60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:15.457106 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd838b98-49a7-4590-99e4-1442de35a60e None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:15.457379 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd838b98-49a7-4590-99e4-1442de35a60e None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:15.458208 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd838b98-49a7-4590-99e4-1442de35a60e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:15.467958 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd838b98-49a7-4590-99e4-1442de35a60e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:03:15.472704 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-dd838b98-49a7-4590-99e4-1442de35a60e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:03:15.472704 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:15.472704 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:03:15.472704 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:03:15.472704 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:03:15.472704 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:03:15.472704 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:03:15.472704 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:03:15.472704 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:03:15.472704 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:03:15.472704 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:03:15.472704 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:03:15.472704 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:03:15.472704 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:03:15.472704 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:03:15.472704 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:03:15.474436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:03:15.474436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:03:15.474436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:03:15.474436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:03:15.474436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:03:15.474436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:03:15.474436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:03:15.474436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:03:15.474436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:03:15.474436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:03:15.474436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:03:15.474436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:03:15.474436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:03:15.474436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:03:15.474436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:03:15.474436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:03:15.475317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:03:15.475317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:03:15.475317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:03:15.475317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:03:15.475317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:03:15.475317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:03:15.475317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:03:15.475317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:03:15.475317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:03:15.475317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:03:15.475317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:03:15.475317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:03:15.475317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 641/1276] 10.222.0.73 () {56 vars in 1096 bytes} [Mon Aug 29 20:03:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:03:15.480465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be030c-6306-41bd-877c-3d7252b654ed None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:15.559749 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:15.578659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be030c-6306-41bd-877c-3d7252b654ed None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:15.613593 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85993057-dd34-4122-ba99-f6e2bb1107f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:15.614437 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85993057-dd34-4122-ba99-f6e2bb1107f8 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:15.614825 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85993057-dd34-4122-ba99-f6e2bb1107f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:15.615097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85993057-dd34-4122-ba99-f6e2bb1107f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:15.615757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85993057-dd34-4122-ba99-f6e2bb1107f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:15.616058 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85993057-dd34-4122-ba99-f6e2bb1107f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:15.617418 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85993057-dd34-4122-ba99-f6e2bb1107f8 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:15.625333 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 636/1277] 10.222.0.73 () {56 vars in 1099 bytes} [Mon Aug 29 20:03:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:15.631253 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd838b98-49a7-4590-99e4-1442de35a60e None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:15.730052 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd838b98-49a7-4590-99e4-1442de35a60e None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:15.763831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08c3ed-f03a-4e4c-88bb-b244cd71b554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:15.764441 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08c3ed-f03a-4e4c-88bb-b244cd71b554 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:15.764718 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08c3ed-f03a-4e4c-88bb-b244cd71b554 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:15.764990 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08c3ed-f03a-4e4c-88bb-b244cd71b554 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:15.775369 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e08c3ed-f03a-4e4c-88bb-b244cd71b554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:15.776803 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e08c3ed-f03a-4e4c-88bb-b244cd71b554 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:15.791475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-3e08c3ed-f03a-4e4c-88bb-b244cd71b554 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:03:15.791475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:15.791475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:03:15.791475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:03:15.791475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:03:15.791475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:03:15.791475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:03:15.791475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:03:15.791475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:03:15.791475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:03:15.791475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:03:15.791475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:03:15.791475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:03:15.791475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:03:15.791475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:03:15.791475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:03:15.792683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:03:15.792683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:03:15.792683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:03:15.792683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:03:15.792683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:03:15.792683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:03:15.792683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:03:15.792683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:03:15.792683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:03:15.792683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:03:15.792683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:03:15.792683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:03:15.792683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:03:15.792683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:03:15.792683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:03:15.792683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:03:15.792683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:03:15.793891 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:03:15.793891 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:03:15.793891 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:03:15.793891 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 642/1278] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:03:15.800249 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85993057-dd34-4122-ba99-f6e2bb1107f8 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:15.897734 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85993057-dd34-4122-ba99-f6e2bb1107f8 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:15.930917 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9bb0a1-c89d-4263-b2c6-8d0f5dcb7523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:15.931565 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bb0a1-c89d-4263-b2c6-8d0f5dcb7523 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:15.931846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bb0a1-c89d-4263-b2c6-8d0f5dcb7523 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:15.932117 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bb0a1-c89d-4263-b2c6-8d0f5dcb7523 None admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:15.932750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9bb0a1-c89d-4263-b2c6-8d0f5dcb7523 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:15.933053 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9bb0a1-c89d-4263-b2c6-8d0f5dcb7523 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:15.934465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9bb0a1-c89d-4263-b2c6-8d0f5dcb7523 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:15.945999 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 637/1279] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:15.951755 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08c3ed-f03a-4e4c-88bb-b244cd71b554 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:15.982418 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:16.050751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08c3ed-f03a-4e4c-88bb-b244cd71b554 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:16.082895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2abd68-9105-4140-9363-3d716d32c6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:16.083728 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2abd68-9105-4140-9363-3d716d32c6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:16.084007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2abd68-9105-4140-9363-3d716d32c6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:16.084276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2abd68-9105-4140-9363-3d716d32c6c4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:16.091759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2abd68-9105-4140-9363-3d716d32c6c4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:16.093195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2abd68-9105-4140-9363-3d716d32c6c4 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:16.104887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-bb2abd68-9105-4140-9363-3d716d32c6c4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:03:16.104887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:16.104887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:03:16.104887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:03:16.104887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:03:16.104887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:03:16.104887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:03:16.104887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:03:16.104887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:03:16.104887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:03:16.104887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:03:16.104887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:03:16.104887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:03:16.104887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:03:16.104887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:03:16.104887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:03:16.106030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:03:16.106030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:03:16.106030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:03:16.106030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:03:16.106030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:03:16.106030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:03:16.106030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:03:16.106030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:03:16.106030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:03:16.106030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:03:16.106030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:03:16.106030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:03:16.106030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:03:16.106030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:03:16.106030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:03:16.106030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:03:16.106030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:03:16.106030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:03:16.107228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:03:16.107228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:03:16.107228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:03:16.107228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:03:16.107228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:03:16.107228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:03:16.107228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:03:16.107228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:03:16.107228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:03:16.107228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:03:16.107228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 643/1280] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:03:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:03:16.111908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9bb0a1-c89d-4263-b2c6-8d0f5dcb7523 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:16.116326 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:16.209687 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9bb0a1-c89d-4263-b2c6-8d0f5dcb7523 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:16.242533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c8ca6-80dd-42d2-aeea-8452e4071cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:16.243123 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c8ca6-80dd-42d2-aeea-8452e4071cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:16.243403 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c8ca6-80dd-42d2-aeea-8452e4071cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:16.243674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c8ca6-80dd-42d2-aeea-8452e4071cc6 None admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:16.244388 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6c8ca6-80dd-42d2-aeea-8452e4071cc6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:16.244693 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6c8ca6-80dd-42d2-aeea-8452e4071cc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:16.246137 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6c8ca6-80dd-42d2-aeea-8452e4071cc6 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:16.262676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 638/1281] 10.222.0.73 () {56 vars in 1143 bytes} [Mon Aug 29 20:03:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:16.268663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2abd68-9105-4140-9363-3d716d32c6c4 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:16.357624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:16.367291 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2abd68-9105-4140-9363-3d716d32c6c4 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:16.400440 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75ad36-0791-4d97-8543-481bd9270eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:16.401109 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75ad36-0791-4d97-8543-481bd9270eaa None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:16.401346 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75ad36-0791-4d97-8543-481bd9270eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:16.401658 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75ad36-0791-4d97-8543-481bd9270eaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:16.470291 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75ad36-0791-4d97-8543-481bd9270eaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:16.471660 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75ad36-0791-4d97-8543-481bd9270eaa None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:16.557568 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:16.568350 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 644/1282] 10.222.0.73 () {58 vars in 1282 bytes} [Mon Aug 29 20:03:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:03:16.574198 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c8ca6-80dd-42d2-aeea-8452e4071cc6 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:16.633933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:16.671683 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c8ca6-80dd-42d2-aeea-8452e4071cc6 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:16.704243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71af205-5734-48e5-a55f-d7d8c13d0a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:16.704851 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71af205-5734-48e5-a55f-d7d8c13d0a90 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:16.705135 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71af205-5734-48e5-a55f-d7d8c13d0a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:16.705405 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71af205-5734-48e5-a55f-d7d8c13d0a90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:16.715608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71af205-5734-48e5-a55f-d7d8c13d0a90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:16.717035 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71af205-5734-48e5-a55f-d7d8c13d0a90 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:16.731156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-d71af205-5734-48e5-a55f-d7d8c13d0a90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:03:16.731156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:16.731156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:03:16.731156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:03:16.731156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:03:16.731156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:03:16.731156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:03:16.731156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:03:16.731156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:03:16.731156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:03:16.731156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:03:16.731156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:03:16.731156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:03:16.731156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:03:16.731156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:03:16.731156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:03:16.732227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:03:16.732227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:03:16.732227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:03:16.732227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:03:16.732227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:03:16.732227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:03:16.732227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:03:16.732227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:03:16.732227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:03:16.732227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:03:16.732227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:03:16.732227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:03:16.732227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:03:16.732227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:03:16.732227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:03:16.732227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:03:16.732227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:03:16.733379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:03:16.733379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:03:16.733379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:03:16.733379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 639/1283] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:03:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:03:16.737884 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75ad36-0791-4d97-8543-481bd9270eaa None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:16.837133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75ad36-0791-4d97-8543-481bd9270eaa None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:16.869501 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77c1d4-d553-4f70-b589-6e262902a7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:16.870116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77c1d4-d553-4f70-b589-6e262902a7e7 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:16.870393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77c1d4-d553-4f70-b589-6e262902a7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:16.870663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77c1d4-d553-4f70-b589-6e262902a7e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:16.871291 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77c1d4-d553-4f70-b589-6e262902a7e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:16.871590 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77c1d4-d553-4f70-b589-6e262902a7e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:16.872944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77c1d4-d553-4f70-b589-6e262902a7e7 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:16.884921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 645/1284] 10.222.0.73 () {56 vars in 1109 bytes} [Mon Aug 29 20:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:16.890948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71af205-5734-48e5-a55f-d7d8c13d0a90 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:16.991077 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71af205-5734-48e5-a55f-d7d8c13d0a90 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:17.004008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:17.024985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063daa44-20aa-4cfd-bf87-031265ca7d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:17.025605 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063daa44-20aa-4cfd-bf87-031265ca7d1b None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:17.025949 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063daa44-20aa-4cfd-bf87-031265ca7d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:17.026224 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063daa44-20aa-4cfd-bf87-031265ca7d1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:17.032638 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063daa44-20aa-4cfd-bf87-031265ca7d1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:17.034098 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063daa44-20aa-4cfd-bf87-031265ca7d1b None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:17.045178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application [None req-063daa44-20aa-4cfd-bf87-031265ca7d1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:03:17.045178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:17.045178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:03:17.045178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:03:17.045178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:03:17.045178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:03:17.045178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:03:17.045178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:03:17.045178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:03:17.045178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:03:17.045178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:03:17.045178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:03:17.045178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:03:17.045178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:03:17.045178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:03:17.046259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:03:17.046259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:03:17.046259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:03:17.046259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:03:17.046259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:03:17.046259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:03:17.046259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:03:17.046259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:03:17.046259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application with Lock( Aug 29 20:03:17.046259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:03:17.046259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:03:17.046259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:03:17.046259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:03:17.046259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:03:17.046259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:03:17.046259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:03:17.046259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:03:17.047213 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:03:17.047213 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:03:17.047213 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:03:17.047213 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:03:17.047213 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:03:17.047213 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:03:17.047213 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:03:17.047213 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: ERROR keystone.server.flask.application Aug 29 20:03:17.047213 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 640/1285] 10.222.0.73 () {56 vars in 1111 bytes} [Mon Aug 29 20:03:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:03:17.052904 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77c1d4-d553-4f70-b589-6e262902a7e7 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:17.103565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:17.153159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77c1d4-d553-4f70-b589-6e262902a7e7 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:17.185843 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296c664-16a0-491a-b64c-adc0d0966b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:17.186374 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296c664-16a0-491a-b64c-adc0d0966b1f None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:17.186651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296c664-16a0-491a-b64c-adc0d0966b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:17.186919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296c664-16a0-491a-b64c-adc0d0966b1f None admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:17.187587 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296c664-16a0-491a-b64c-adc0d0966b1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:17.187889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296c664-16a0-491a-b64c-adc0d0966b1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:17.189293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296c664-16a0-491a-b64c-adc0d0966b1f None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:17.201428 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 646/1286] 10.222.0.73 () {56 vars in 1148 bytes} [Mon Aug 29 20:03:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:17.207575 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063daa44-20aa-4cfd-bf87-031265ca7d1b None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:17.306549 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063daa44-20aa-4cfd-bf87-031265ca7d1b None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:17.338038 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf025177-6841-445f-8dcc-ede27c09c4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:17.338625 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf025177-6841-445f-8dcc-ede27c09c4ea None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:17.338901 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf025177-6841-445f-8dcc-ede27c09c4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:17.339171 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf025177-6841-445f-8dcc-ede27c09c4ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:17.339883 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf025177-6841-445f-8dcc-ede27c09c4ea None admin] RBAC: Adding query filter params (role.id=296ee2c194824485af71618619f82e09, scope.project.id=2f5ec615d98548159487c97cead336a0, user.id=1ab2e318b2854c46bc7ed2151a44a2e2) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:17.340188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf025177-6841-445f-8dcc-ede27c09c4ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:17.341602 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf025177-6841-445f-8dcc-ede27c09c4ea None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:17.347292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 641/1287] 10.222.0.73 () {56 vars in 1374 bytes} [Mon Aug 29 20:03:17 2022] GET /identity/v3/role_assignments?user.id=1ab2e318b2854c46bc7ed2151a44a2e2&scope.project.id=2f5ec615d98548159487c97cead336a0&role.id=296ee2c194824485af71618619f82e09 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:19.246228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8737cd7-6b1b-4272-845f-6daf61e9527d None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:19.246780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8737cd7-6b1b-4272-845f-6daf61e9527d None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:19.246780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8737cd7-6b1b-4272-845f-6daf61e9527d None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:19.248087 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 647/1288] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:03:19.253064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e6c1f-4caf-4985-9d83-1eda89d3427a None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:19.253482 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e6c1f-4caf-4985-9d83-1eda89d3427a None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:19.253619 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e6c1f-4caf-4985-9d83-1eda89d3427a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:19.272946 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-af9e6c1f-4caf-4985-9d83-1eda89d3427a None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:03:19.274161 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-af9e6c1f-4caf-4985-9d83-1eda89d3427a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:03:19.303704 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:19.340834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 642/1289] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:03:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:03:19.608987 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8737cd7-6b1b-4272-845f-6daf61e9527d None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:19.708441 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8737cd7-6b1b-4272-845f-6daf61e9527d None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:19.718834 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:19.742633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59fe00-e9a7-4d3b-8bf0-0b1faf69ac41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:19.743244 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59fe00-e9a7-4d3b-8bf0-0b1faf69ac41 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:19.743521 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59fe00-e9a7-4d3b-8bf0-0b1faf69ac41 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:19.743793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59fe00-e9a7-4d3b-8bf0-0b1faf69ac41 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:19.744366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c59fe00-e9a7-4d3b-8bf0-0b1faf69ac41 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:19.745750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c59fe00-e9a7-4d3b-8bf0-0b1faf69ac41 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:19.754276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-8c59fe00-e9a7-4d3b-8bf0-0b1faf69ac41 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:03:19.754276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:19.754276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:03:19.754276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:03:19.754276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:03:19.754276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:03:19.754276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:03:19.754276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:03:19.754276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:03:19.754276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:03:19.754276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:03:19.754276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:03:19.754276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:19.754276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:03:19.754276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:03:19.754276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:03:19.754276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:03:19.755819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:03:19.755819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:03:19.755819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:03:19.755819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:03:19.755819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:03:19.755819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:03:19.755819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:03:19.755819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:03:19.755819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:03:19.755819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:03:19.755819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:03:19.755819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:03:19.755819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:03:19.755819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:03:19.755819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:03:19.755819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:03:19.757002 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:03:19.757002 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:03:19.757002 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:03:19.757002 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:03:19.757002 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:03:19.757002 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:03:19.757002 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:03:19.757002 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:03:19.757002 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:03:19.757002 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:03:19.757002 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:03:19.757002 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:03:19.757002 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:03:19.757002 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:03:19.757002 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 648/1290] 10.222.0.73 () {56 vars in 1117 bytes} [Mon Aug 29 20:03:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:03:19.761514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e6c1f-4caf-4985-9d83-1eda89d3427a None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:19.857255 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e6c1f-4caf-4985-9d83-1eda89d3427a None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:19.890615 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b44c0-6aae-4294-9005-8f8397f7ad8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:19.891152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b44c0-6aae-4294-9005-8f8397f7ad8f None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:19.891434 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b44c0-6aae-4294-9005-8f8397f7ad8f None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:19.891709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b44c0-6aae-4294-9005-8f8397f7ad8f None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:19.892334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b44c0-6aae-4294-9005-8f8397f7ad8f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:19.892639 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b44c0-6aae-4294-9005-8f8397f7ad8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:19.894031 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b44c0-6aae-4294-9005-8f8397f7ad8f None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:19.899288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 643/1291] 10.222.0.73 () {56 vars in 1116 bytes} [Mon Aug 29 20:03:19 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:19.904925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59fe00-e9a7-4d3b-8bf0-0b1faf69ac41 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:20.002426 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59fe00-e9a7-4d3b-8bf0-0b1faf69ac41 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:20.035205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522482ae-df31-4790-8d9a-e84fc492d9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:20.035797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522482ae-df31-4790-8d9a-e84fc492d9c2 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:20.036659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522482ae-df31-4790-8d9a-e84fc492d9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:20.036937 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522482ae-df31-4790-8d9a-e84fc492d9c2 None admin] PATH_INFO: `/v3/services` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:20.037564 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522482ae-df31-4790-8d9a-e84fc492d9c2 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:20.037896 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522482ae-df31-4790-8d9a-e84fc492d9c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:20.039266 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522482ae-df31-4790-8d9a-e84fc492d9c2 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:20.043583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 649/1292] 10.222.0.73 () {56 vars in 1116 bytes} [Mon Aug 29 20:03:19 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:21.940488 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec096544-6df1-4594-9f27-867dd486bfca None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:21.941016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec096544-6df1-4594-9f27-867dd486bfca None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:21.941105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec096544-6df1-4594-9f27-867dd486bfca None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:21.943286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 644/1293] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:03:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:03:21.948488 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c12e1-1403-437f-91ae-1e8f70eea2e6 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:21.948867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c12e1-1403-437f-91ae-1e8f70eea2e6 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:21.949031 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c12e1-1403-437f-91ae-1e8f70eea2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:21.967429 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-ec6c12e1-1403-437f-91ae-1e8f70eea2e6 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:03:21.968602 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ec6c12e1-1403-437f-91ae-1e8f70eea2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:03:21.996932 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:22.034015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 650/1294] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:03:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:03:22.298533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec096544-6df1-4594-9f27-867dd486bfca None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:22.329320 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:22.402113 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec096544-6df1-4594-9f27-867dd486bfca None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:22.443303 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875e027-3768-40ca-85b6-d9d9b924f37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:22.444239 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875e027-3768-40ca-85b6-d9d9b924f37e None admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:22.444653 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875e027-3768-40ca-85b6-d9d9b924f37e None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:22.445119 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875e027-3768-40ca-85b6-d9d9b924f37e None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:22.446172 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875e027-3768-40ca-85b6-d9d9b924f37e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:22.448047 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875e027-3768-40ca-85b6-d9d9b924f37e None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:22.467751 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 645/1295] 10.222.0.73 () {60 vars in 1141 bytes} [Mon Aug 29 20:03:22 2022] POST /identity/v3/services => generated 246 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:03:24.367987 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da54c3-5ebb-4df0-85c7-b7bf0385069c None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:24.368514 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da54c3-5ebb-4df0-85c7-b7bf0385069c None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:24.368514 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da54c3-5ebb-4df0-85c7-b7bf0385069c None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:24.369811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 651/1296] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:03:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:03:24.374867 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddd4b0-53ce-40b2-9902-a1d3ebea372f None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:24.375290 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddd4b0-53ce-40b2-9902-a1d3ebea372f None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:24.375414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddd4b0-53ce-40b2-9902-a1d3ebea372f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:24.394223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-0dddd4b0-53ce-40b2-9902-a1d3ebea372f None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:03:24.395385 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-0dddd4b0-53ce-40b2-9902-a1d3ebea372f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:03:24.467485 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 646/1297] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:03:24 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:03:24.734267 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da54c3-5ebb-4df0-85c7-b7bf0385069c None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:24.837905 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da54c3-5ebb-4df0-85c7-b7bf0385069c None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:24.870624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f90f7-d593-4839-944e-f147c1c881f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:24.871236 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f90f7-d593-4839-944e-f147c1c881f4 None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:24.871517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f90f7-d593-4839-944e-f147c1c881f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:24.871787 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f90f7-d593-4839-944e-f147c1c881f4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:24.872359 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f90f7-d593-4839-944e-f147c1c881f4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:24.873737 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f90f7-d593-4839-944e-f147c1c881f4 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:24.882889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-616f90f7-d593-4839-944e-f147c1c881f4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:03:24.882889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:24.882889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:03:24.882889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:03:24.882889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:03:24.882889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:03:24.882889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:03:24.882889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:03:24.882889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:03:24.882889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:03:24.882889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:03:24.882889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:03:24.882889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:24.882889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:03:24.882889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:03:24.882889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:03:24.882889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:03:24.884116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:03:24.884116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:03:24.884116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:03:24.884116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:03:24.884116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:03:24.884116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:03:24.884116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:03:24.884116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:03:24.884116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:03:24.884116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:03:24.884116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:03:24.884116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:03:24.884116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:03:24.884116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:03:24.884116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:03:24.884116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:03:24.885230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:03:24.885230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:03:24.885230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:03:24.885230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:03:24.885230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:03:24.885230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:03:24.885230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:03:24.885230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:03:24.885230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:03:24.885230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:03:24.885230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:03:24.885230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:03:24.885230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:03:24.885230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:03:24.885230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 652/1298] 10.222.0.73 () {56 vars in 1117 bytes} [Mon Aug 29 20:03:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:03:24.890329 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddd4b0-53ce-40b2-9902-a1d3ebea372f None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:24.908753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:24.987311 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddd4b0-53ce-40b2-9902-a1d3ebea372f None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:25.020061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8de8fc-e38e-406a-9ccd-a672efb4ae98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:25.020784 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8de8fc-e38e-406a-9ccd-a672efb4ae98 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:25.021158 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8de8fc-e38e-406a-9ccd-a672efb4ae98 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:25.021430 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8de8fc-e38e-406a-9ccd-a672efb4ae98 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:25.022076 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8de8fc-e38e-406a-9ccd-a672efb4ae98 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:25.022375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8de8fc-e38e-406a-9ccd-a672efb4ae98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:25.023726 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8de8fc-e38e-406a-9ccd-a672efb4ae98 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:25.028694 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 647/1299] 10.222.0.73 () {56 vars in 1116 bytes} [Mon Aug 29 20:03:24 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:25.034765 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f90f7-d593-4839-944e-f147c1c881f4 None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:25.066560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:25.136389 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f90f7-d593-4839-944e-f147c1c881f4 None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:25.170199 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21386396-0880-46c5-8bf8-466f37dc913d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:25.170792 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21386396-0880-46c5-8bf8-466f37dc913d None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:25.171071 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21386396-0880-46c5-8bf8-466f37dc913d None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:25.171343 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21386396-0880-46c5-8bf8-466f37dc913d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:25.172034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21386396-0880-46c5-8bf8-466f37dc913d None admin] RBAC: Adding query filter params (interface=public, service_id=1d7d11b3841e46c580e7480cfd038be4, region_id=RegionOne) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:25.172358 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21386396-0880-46c5-8bf8-466f37dc913d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:25.173689 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21386396-0880-46c5-8bf8-466f37dc913d None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:25.178844 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 653/1300] 10.222.0.73 () {56 vars in 1251 bytes} [Mon Aug 29 20:03:25 2022] GET /identity/v3/endpoints?service_id=1d7d11b3841e46c580e7480cfd038be4&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:03:25.184732 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8de8fc-e38e-406a-9ccd-a672efb4ae98 None admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:25.266107 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:25.285260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8de8fc-e38e-406a-9ccd-a672efb4ae98 None admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:25.319940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbd414-9984-410d-801b-32fe1e29d401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:25.320552 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbd414-9984-410d-801b-32fe1e29d401 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:25.320919 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbd414-9984-410d-801b-32fe1e29d401 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:25.321197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbd414-9984-410d-801b-32fe1e29d401 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:25.321840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbd414-9984-410d-801b-32fe1e29d401 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:25.323204 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbd414-9984-410d-801b-32fe1e29d401 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:25.331201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 648/1301] 10.222.0.73 () {56 vars in 1087 bytes} [Mon Aug 29 20:03:25 2022] GET /identity/v3/services => generated 2191 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:03:27.336910 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb27cf9-e094-4eb5-bdfd-0c681f5acb6e None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:27.337455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb27cf9-e094-4eb5-bdfd-0c681f5acb6e None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:27.337455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb27cf9-e094-4eb5-bdfd-0c681f5acb6e None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:27.338893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 654/1302] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:03:27.343947 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944089e-ece1-48d1-b19c-08c676a1ef87 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:27.344353 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944089e-ece1-48d1-b19c-08c676a1ef87 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:27.344502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944089e-ece1-48d1-b19c-08c676a1ef87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:27.362492 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-5944089e-ece1-48d1-b19c-08c676a1ef87 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:03:27.363676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-5944089e-ece1-48d1-b19c-08c676a1ef87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:03:27.441926 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 649/1303] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:03:27 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:03:27.709528 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb27cf9-e094-4eb5-bdfd-0c681f5acb6e None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:27.804340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:27.811390 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb27cf9-e094-4eb5-bdfd-0c681f5acb6e None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:27.843889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc507f-e1fe-4cc4-b0b0-dfd5e7266d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:27.844502 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc507f-e1fe-4cc4-b0b0-dfd5e7266d7c None admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:27.844777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc507f-e1fe-4cc4-b0b0-dfd5e7266d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:27.845045 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc507f-e1fe-4cc4-b0b0-dfd5e7266d7c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:27.845611 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc507f-e1fe-4cc4-b0b0-dfd5e7266d7c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:27.846992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc507f-e1fe-4cc4-b0b0-dfd5e7266d7c None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:27.856473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application [None req-d4cc507f-e1fe-4cc4-b0b0-dfd5e7266d7c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:03:27.856473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:27.856473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:03:27.856473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:03:27.856473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:03:27.856473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:03:27.856473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:03:27.856473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:03:27.856473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:03:27.856473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:03:27.856473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:03:27.856473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:03:27.856473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:03:27.856473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:03:27.856473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:03:27.856473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:03:27.856473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:03:27.857638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:03:27.857638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:03:27.857638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:03:27.857638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:03:27.857638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:03:27.857638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:03:27.857638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:03:27.857638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:03:27.857638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:03:27.857638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:03:27.857638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:03:27.857638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:03:27.857638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:03:27.857638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:03:27.857638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:03:27.857638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:03:27.858899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:03:27.858899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application with Lock( Aug 29 20:03:27.858899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:03:27.858899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:03:27.858899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:03:27.858899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:03:27.858899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:03:27.858899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:03:27.858899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:03:27.858899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:03:27.858899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:03:27.858899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:03:27.858899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:03:27.858899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: ERROR keystone.server.flask.application Aug 29 20:03:27.858899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 655/1304] 10.222.0.73 () {56 vars in 1117 bytes} [Mon Aug 29 20:03:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:03:27.863537 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944089e-ece1-48d1-b19c-08c676a1ef87 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:27.938103 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:27.960563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944089e-ece1-48d1-b19c-08c676a1ef87 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:27.994044 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ea4ef-b9bc-43bb-865a-3d438f6c17f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:27.994641 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ea4ef-b9bc-43bb-865a-3d438f6c17f1 None admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:27.994921 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ea4ef-b9bc-43bb-865a-3d438f6c17f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:27.995194 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ea4ef-b9bc-43bb-865a-3d438f6c17f1 None admin] PATH_INFO: `/v3/services` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:27.995819 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ea4ef-b9bc-43bb-865a-3d438f6c17f1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:03:27.996125 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ea4ef-b9bc-43bb-865a-3d438f6c17f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:27.997475 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ea4ef-b9bc-43bb-865a-3d438f6c17f1 None admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:28.002515 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 650/1305] 10.222.0.73 () {56 vars in 1116 bytes} [Mon Aug 29 20:03:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:03:28.008271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc507f-e1fe-4cc4-b0b0-dfd5e7266d7c None admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:03:28.108545 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc507f-e1fe-4cc4-b0b0-dfd5e7266d7c None admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:03:28.141253 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca134a42-b544-4982-9b59-4f9fe63d39b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:03:28.141872 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca134a42-b544-4982-9b59-4f9fe63d39b2 None admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:03:28.142147 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca134a42-b544-4982-9b59-4f9fe63d39b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:03:28.142413 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca134a42-b544-4982-9b59-4f9fe63d39b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:03:28.143068 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca134a42-b544-4982-9b59-4f9fe63d39b2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:03:28.144416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca134a42-b544-4982-9b59-4f9fe63d39b2 None admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:03:28.163899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 656/1306] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:03:28 2022] POST /identity/v3/endpoints => generated 331 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:03:37.999540 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=30 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:03:38.159139 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:34.448847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441250e6-bf9b-4b8a-969d-af300403bcde None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:34.449410 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441250e6-bf9b-4b8a-969d-af300403bcde None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:34.449410 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441250e6-bf9b-4b8a-969d-af300403bcde None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:34.456005 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b26906-009d-4710-96f6-06588ead2104 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:34.456461 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b26906-009d-4710-96f6-06588ead2104 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:34.456557 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b26906-009d-4710-96f6-06588ead2104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:34.460407 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 651/1307] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:04:34 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:34.538281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-04b26906-009d-4710-96f6-06588ead2104 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:34.540056 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-04b26906-009d-4710-96f6-06588ead2104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:34.670989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 657/1308] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:04:34.993592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8a356-40a9-4cc9-aa05-93621d9673a4 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:34.994108 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8a356-40a9-4cc9-aa05-93621d9673a4 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:34.994471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8a356-40a9-4cc9-aa05-93621d9673a4 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:34.996654 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 652/1309] 10.222.0.73 () {54 vars in 965 bytes} [Mon Aug 29 20:04:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:35.004075 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1477c00-e154-424d-9b08-5bea1d69b0d1 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:35.004421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1477c00-e154-424d-9b08-5bea1d69b0d1 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:35.004612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1477c00-e154-424d-9b08-5bea1d69b0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:35.124501 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-e1477c00-e154-424d-9b08-5bea1d69b0d1 None None] MFA Rules not processed for user `64ba58d774674ed7b233ad6416db068a`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:35.125739 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-e1477c00-e154-424d-9b08-5bea1d69b0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:35.150126 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:35.217649 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 658/1310] 10.222.0.73 () {58 vars in 1062 bytes} [Mon Aug 29 20:04:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:04:35.227089 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8a356-40a9-4cc9-aa05-93621d9673a4 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:35.229187 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-22b8a356-40a9-4cc9-aa05-93621d9673a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:35.363707 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8a356-40a9-4cc9-aa05-93621d9673a4 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:35.455953 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0cd7a-ffe3-4252-8cd8-c85c271d69de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:35.456592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0cd7a-ffe3-4252-8cd8-c85c271d69de service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:35.456833 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0cd7a-ffe3-4252-8cd8-c85c271d69de service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:35.457018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0cd7a-ffe3-4252-8cd8-c85c271d69de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:35.458770 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-4fa0cd7a-ffe3-4252-8cd8-c85c271d69de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:35.530106 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:35.603525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0cd7a-ffe3-4252-8cd8-c85c271d69de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:35.604996 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0cd7a-ffe3-4252-8cd8-c85c271d69de service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:35.627325 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 653/1311] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:04:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:04:38.283838 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce9440-e03d-47c5-96f0-3d3d97671147 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:38.284403 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce9440-e03d-47c5-96f0-3d3d97671147 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:38.284494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce9440-e03d-47c5-96f0-3d3d97671147 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:38.286250 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 659/1312] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:04:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:38.295043 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295cdd6-89cc-40d3-ba1f-eb52e27827cd None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:38.295792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295cdd6-89cc-40d3-ba1f-eb52e27827cd None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:38.295992 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295cdd6-89cc-40d3-ba1f-eb52e27827cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:38.428050 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-f295cdd6-89cc-40d3-ba1f-eb52e27827cd None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:38.429266 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-f295cdd6-89cc-40d3-ba1f-eb52e27827cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:38.581516 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 654/1313] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:04:38.895904 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f276ae8-d5b2-43fb-87f3-2dcee8ce1803 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:38.896186 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f276ae8-d5b2-43fb-87f3-2dcee8ce1803 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:38.896462 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f276ae8-d5b2-43fb-87f3-2dcee8ce1803 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:38.897855 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 660/1314] 10.222.0.73 () {54 vars in 965 bytes} [Mon Aug 29 20:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:38.904037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bba5a-2748-42ea-ad46-4a7c1d27c956 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:38.904329 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bba5a-2748-42ea-ad46-4a7c1d27c956 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:38.904609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bba5a-2748-42ea-ad46-4a7c1d27c956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:38.917183 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:38.984870 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-2e5bba5a-2748-42ea-ad46-4a7c1d27c956 None None] MFA Rules not processed for user `64ba58d774674ed7b233ad6416db068a`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:38.986029 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-2e5bba5a-2748-42ea-ad46-4a7c1d27c956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:39.065086 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 655/1315] 10.222.0.73 () {58 vars in 1062 bytes} [Mon Aug 29 20:04:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:04:39.074390 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f276ae8-d5b2-43fb-87f3-2dcee8ce1803 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:39.076184 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-0f276ae8-d5b2-43fb-87f3-2dcee8ce1803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:39.191051 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f276ae8-d5b2-43fb-87f3-2dcee8ce1803 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:39.263938 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:39.266455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccafcd3-3f81-4364-a487-f09c0964c851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:39.267093 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccafcd3-3f81-4364-a487-f09c0964c851 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:39.267582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccafcd3-3f81-4364-a487-f09c0964c851 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:39.268021 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccafcd3-3f81-4364-a487-f09c0964c851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:39.270132 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-4ccafcd3-3f81-4364-a487-f09c0964c851 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:39.395105 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccafcd3-3f81-4364-a487-f09c0964c851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:39.396552 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccafcd3-3f81-4364-a487-f09c0964c851 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:39.417994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 661/1316] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:04:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:04:42.029031 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca7485-4427-4630-aae8-e7e66aa1d5ea None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:42.029568 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca7485-4427-4630-aae8-e7e66aa1d5ea None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:42.029568 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca7485-4427-4630-aae8-e7e66aa1d5ea None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:42.030987 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 656/1317] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:42.036075 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed7b47-061d-44f6-91cd-ec13a876f808 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:42.036518 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed7b47-061d-44f6-91cd-ec13a876f808 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:42.036625 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed7b47-061d-44f6-91cd-ec13a876f808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:42.104244 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-b8ed7b47-061d-44f6-91cd-ec13a876f808 None None] MFA Rules not processed for user `b821a3d0f6354acaad7a12c771fb8141`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:42.105446 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-b8ed7b47-061d-44f6-91cd-ec13a876f808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:42.168795 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:42.219102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 662/1318] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:42 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:04:42.511729 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca7485-4427-4630-aae8-e7e66aa1d5ea None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:42.513377 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-f1ca7485-4427-4630-aae8-e7e66aa1d5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:42.627445 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca7485-4427-4630-aae8-e7e66aa1d5ea None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:42.630420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:42.733327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82554cd-e92a-4871-8e29-b01ea03b990f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:42.733979 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82554cd-e92a-4871-8e29-b01ea03b990f service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:42.734260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82554cd-e92a-4871-8e29-b01ea03b990f service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:42.734530 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82554cd-e92a-4871-8e29-b01ea03b990f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:42.736275 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c82554cd-e92a-4871-8e29-b01ea03b990f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:42.849854 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82554cd-e92a-4871-8e29-b01ea03b990f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:42.851293 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82554cd-e92a-4871-8e29-b01ea03b990f service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:42.878321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 657/1319] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:04:42 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:04:45.374688 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739982c-bc42-4bdd-83f4-2f968d20b792 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:45.375198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739982c-bc42-4bdd-83f4-2f968d20b792 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:45.375198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739982c-bc42-4bdd-83f4-2f968d20b792 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:45.376529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 663/1320] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:45.381802 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77e15c-4e79-4500-9b22-33f43fe1950f None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:45.382224 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77e15c-4e79-4500-9b22-33f43fe1950f None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:45.382326 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77e15c-4e79-4500-9b22-33f43fe1950f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:45.424737 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:45.449695 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-7a77e15c-4e79-4500-9b22-33f43fe1950f None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:45.450959 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-7a77e15c-4e79-4500-9b22-33f43fe1950f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:45.549233 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 658/1321] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:04:45.842022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739982c-bc42-4bdd-83f4-2f968d20b792 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:45.843578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a739982c-bc42-4bdd-83f4-2f968d20b792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:45.947276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739982c-bc42-4bdd-83f4-2f968d20b792 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:46.020857 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a02a5-0cf3-4b00-a8fe-4f9e27c689a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:46.021471 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a02a5-0cf3-4b00-a8fe-4f9e27c689a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:46.021772 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a02a5-0cf3-4b00-a8fe-4f9e27c689a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:46.022045 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a02a5-0cf3-4b00-a8fe-4f9e27c689a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:46.023672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-314a02a5-0cf3-4b00-a8fe-4f9e27c689a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:46.032670 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:46.135903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a02a5-0cf3-4b00-a8fe-4f9e27c689a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:46.137346 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a02a5-0cf3-4b00-a8fe-4f9e27c689a2 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:46.159356 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 664/1322] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:04:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:04:48.845267 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977782a-5184-432b-9e88-8bbd0a71dfd6 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:48.845816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977782a-5184-432b-9e88-8bbd0a71dfd6 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:48.845935 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977782a-5184-432b-9e88-8bbd0a71dfd6 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:48.847205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 659/1323] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:48.852266 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2144e8-30aa-4f2b-bfa9-9e4809aa401b None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:48.852831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2144e8-30aa-4f2b-bfa9-9e4809aa401b None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:48.853110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2144e8-30aa-4f2b-bfa9-9e4809aa401b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:48.921600 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-1a2144e8-30aa-4f2b-bfa9-9e4809aa401b None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:48.922776 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-1a2144e8-30aa-4f2b-bfa9-9e4809aa401b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:48.981979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:49.022842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 665/1324] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:04:49.312156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977782a-5184-432b-9e88-8bbd0a71dfd6 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:49.313721 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-6977782a-5184-432b-9e88-8bbd0a71dfd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:49.417542 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977782a-5184-432b-9e88-8bbd0a71dfd6 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:49.429490 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:49.494980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0164038-06a5-42bb-b64b-c94dd49560da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:49.495480 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0164038-06a5-42bb-b64b-c94dd49560da service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:49.495761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0164038-06a5-42bb-b64b-c94dd49560da service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:49.496030 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0164038-06a5-42bb-b64b-c94dd49560da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:49.497659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d0164038-06a5-42bb-b64b-c94dd49560da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:49.615995 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0164038-06a5-42bb-b64b-c94dd49560da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:49.619079 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0164038-06a5-42bb-b64b-c94dd49560da service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:49.638005 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 660/1325] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:04:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:04:51.607957 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ae66b-1c51-46ce-84d3-1ce64ccb55cf None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:51.608487 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ae66b-1c51-46ce-84d3-1ce64ccb55cf None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:51.608487 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ae66b-1c51-46ce-84d3-1ce64ccb55cf None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:51.609864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 666/1326] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:51.615007 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8ae41-d641-4681-b0d0-0d9eb55db71a None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:51.615448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8ae41-d641-4681-b0d0-0d9eb55db71a None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:51.615680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8ae41-d641-4681-b0d0-0d9eb55db71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:51.665886 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:51.685100 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-e2c8ae41-d641-4681-b0d0-0d9eb55db71a None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:51.686345 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-e2c8ae41-d641-4681-b0d0-0d9eb55db71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:51.787324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 661/1327] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:04:52.067616 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ae66b-1c51-46ce-84d3-1ce64ccb55cf None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:52.069204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-3a1ae66b-1c51-46ce-84d3-1ce64ccb55cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:52.184134 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ae66b-1c51-46ce-84d3-1ce64ccb55cf None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:52.261972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72ac1e-ecea-4859-b6ce-b847725182e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:52.262596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72ac1e-ecea-4859-b6ce-b847725182e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:52.262875 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72ac1e-ecea-4859-b6ce-b847725182e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:52.263150 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72ac1e-ecea-4859-b6ce-b847725182e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:52.264760 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ac72ac1e-ecea-4859-b6ce-b847725182e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:52.282836 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:52.376226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72ac1e-ecea-4859-b6ce-b847725182e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:52.377662 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72ac1e-ecea-4859-b6ce-b847725182e1 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:52.396745 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 667/1328] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:04:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:04:54.375096 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dde86d-d9ec-408b-b5ab-52a6c9fb4385 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:54.375641 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dde86d-d9ec-408b-b5ab-52a6c9fb4385 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:54.375641 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dde86d-d9ec-408b-b5ab-52a6c9fb4385 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:54.377029 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 662/1329] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:54.382830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c21d14-257e-419e-a122-6b994d405b61 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:54.383398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c21d14-257e-419e-a122-6b994d405b61 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:54.383398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c21d14-257e-419e-a122-6b994d405b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:54.453902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-e4c21d14-257e-419e-a122-6b994d405b61 None None] MFA Rules not processed for user `b821a3d0f6354acaad7a12c771fb8141`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:54.455111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-e4c21d14-257e-419e-a122-6b994d405b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:54.533771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:54.567308 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 668/1330] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:54 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:04:54.851280 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dde86d-d9ec-408b-b5ab-52a6c9fb4385 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:54.853457 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-43dde86d-d9ec-408b-b5ab-52a6c9fb4385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:54.959214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dde86d-d9ec-408b-b5ab-52a6c9fb4385 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:54.979465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:55.032768 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3e172-7642-423e-8fb5-e790f81d1fd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:55.033295 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3e172-7642-423e-8fb5-e790f81d1fd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:55.033567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3e172-7642-423e-8fb5-e790f81d1fd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:55.033885 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3e172-7642-423e-8fb5-e790f81d1fd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:55.035551 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-fba3e172-7642-423e-8fb5-e790f81d1fd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:55.145251 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba3e172-7642-423e-8fb5-e790f81d1fd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:55.146979 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba3e172-7642-423e-8fb5-e790f81d1fd5 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:55.166249 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 663/1331] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:04:54 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:04:57.328352 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59d6f7-6266-45b5-abff-3ea962beb66e None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:57.328879 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59d6f7-6266-45b5-abff-3ea962beb66e None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:57.328879 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59d6f7-6266-45b5-abff-3ea962beb66e None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:57.330245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 669/1332] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:57.335385 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8a8e1-43f5-43fd-9d3d-65769c4bf1cf None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:57.335799 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8a8e1-43f5-43fd-9d3d-65769c4bf1cf None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:57.335924 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8a8e1-43f5-43fd-9d3d-65769c4bf1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:57.400707 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:57.403404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-dfc8a8e1-43f5-43fd-9d3d-65769c4bf1cf None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:57.405356 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-dfc8a8e1-43f5-43fd-9d3d-65769c4bf1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:57.500193 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 664/1333] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:04:57.779184 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba59d6f7-6266-45b5-abff-3ea962beb66e None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:57.780743 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ba59d6f7-6266-45b5-abff-3ea962beb66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:57.886434 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba59d6f7-6266-45b5-abff-3ea962beb66e None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:57.960119 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dab6e7-a485-445f-a60f-83e6ef51ffe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:57.960727 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dab6e7-a485-445f-a60f-83e6ef51ffe0 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:57.961008 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dab6e7-a485-445f-a60f-83e6ef51ffe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:57.961282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dab6e7-a485-445f-a60f-83e6ef51ffe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:57.963628 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-90dab6e7-a485-445f-a60f-83e6ef51ffe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:57.990893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:58.073402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dab6e7-a485-445f-a60f-83e6ef51ffe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:58.074897 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dab6e7-a485-445f-a60f-83e6ef51ffe0 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:58.093968 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 670/1334] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:04:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:05:00.109764 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741c8c6-b777-4a89-95fd-0a8addeb548f None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:00.110265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741c8c6-b777-4a89-95fd-0a8addeb548f None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:00.110387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741c8c6-b777-4a89-95fd-0a8addeb548f None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:00.111658 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 665/1335] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:00.116726 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4009a-8aa4-49f0-8ffa-c4f015302642 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:00.117146 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4009a-8aa4-49f0-8ffa-c4f015302642 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:00.117275 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4009a-8aa4-49f0-8ffa-c4f015302642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:00.184813 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-7cd4009a-8aa4-49f0-8ffa-c4f015302642 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:00.186081 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-7cd4009a-8aa4-49f0-8ffa-c4f015302642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:00.260405 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:00.283105 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 671/1336] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:00.574653 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741c8c6-b777-4a89-95fd-0a8addeb548f None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:00.576233 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-7741c8c6-b777-4a89-95fd-0a8addeb548f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:00.682791 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741c8c6-b777-4a89-95fd-0a8addeb548f None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:00.709865 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:00.754275 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174cf371-aa8e-4905-8278-b3ebd938ef0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:00.754875 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174cf371-aa8e-4905-8278-b3ebd938ef0c service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:00.755145 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174cf371-aa8e-4905-8278-b3ebd938ef0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:00.755409 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174cf371-aa8e-4905-8278-b3ebd938ef0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:00.756991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-174cf371-aa8e-4905-8278-b3ebd938ef0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:00.863886 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174cf371-aa8e-4905-8278-b3ebd938ef0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:00.865300 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174cf371-aa8e-4905-8278-b3ebd938ef0c service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:00.882713 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 666/1337] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:05:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:05:03.099378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1e4a4-ec23-42c2-9d39-a8745bb2a8f6 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:03.099880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1e4a4-ec23-42c2-9d39-a8745bb2a8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:03.099972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1e4a4-ec23-42c2-9d39-a8745bb2a8f6 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:03.101254 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 672/1338] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:05:03.106967 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd3add-d2a4-424b-b0a1-b11dcfcf2237 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:03.107423 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd3add-d2a4-424b-b0a1-b11dcfcf2237 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:03.107527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd3add-d2a4-424b-b0a1-b11dcfcf2237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:03.172781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-6fdd3add-d2a4-424b-b0a1-b11dcfcf2237 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:03.174005 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-6fdd3add-d2a4-424b-b0a1-b11dcfcf2237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:03.182694 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:03.269222 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 667/1339] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:03.550452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1e4a4-ec23-42c2-9d39-a8745bb2a8f6 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:03.551997 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-37e1e4a4-ec23-42c2-9d39-a8745bb2a8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:03.656224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1e4a4-ec23-42c2-9d39-a8745bb2a8f6 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:03.728004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70d370-0db0-4236-9763-3fd1e353b72a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:03.729121 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70d370-0db0-4236-9763-3fd1e353b72a service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:03.729396 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70d370-0db0-4236-9763-3fd1e353b72a service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:03.729661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70d370-0db0-4236-9763-3fd1e353b72a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:03.731310 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-4e70d370-0db0-4236-9763-3fd1e353b72a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:03.766937 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:03.843529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70d370-0db0-4236-9763-3fd1e353b72a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:03.844980 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70d370-0db0-4236-9763-3fd1e353b72a service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:03.863388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 673/1340] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:05:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:05:07.687438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb29440-e418-431d-a88f-4a33120ad0bc None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:07.687978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb29440-e418-431d-a88f-4a33120ad0bc None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:07.687978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb29440-e418-431d-a88f-4a33120ad0bc None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:07.689324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 668/1341] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:07.694385 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01570f1a-23d9-46f7-baa2-b963184270a9 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:07.694817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01570f1a-23d9-46f7-baa2-b963184270a9 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:07.694910 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01570f1a-23d9-46f7-baa2-b963184270a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:07.761592 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-01570f1a-23d9-46f7-baa2-b963184270a9 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:07.762844 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-01570f1a-23d9-46f7-baa2-b963184270a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:07.852213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:07.864978 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 674/1342] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:08.146332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb29440-e418-431d-a88f-4a33120ad0bc None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:08.148149 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-0eb29440-e418-431d-a88f-4a33120ad0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:08.253461 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb29440-e418-431d-a88f-4a33120ad0bc None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:08.295051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:08.331531 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022fa47a-9f25-406a-b34c-f24cfa0236b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:08.332135 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fa47a-9f25-406a-b34c-f24cfa0236b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:08.332411 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fa47a-9f25-406a-b34c-f24cfa0236b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:08.332678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fa47a-9f25-406a-b34c-f24cfa0236b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:08.334311 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-022fa47a-9f25-406a-b34c-f24cfa0236b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:08.442774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022fa47a-9f25-406a-b34c-f24cfa0236b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:08.444442 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022fa47a-9f25-406a-b34c-f24cfa0236b8 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:08.463880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 669/1343] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:05:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:05:11.035223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bc6ba-0bb5-4087-83a6-cf4cadd2a8e9 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:11.035770 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bc6ba-0bb5-4087-83a6-cf4cadd2a8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:11.035770 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bc6ba-0bb5-4087-83a6-cf4cadd2a8e9 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:11.037077 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 675/1344] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:11.043417 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb26f0-0344-402d-be30-643eddb8b6a9 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:11.043952 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb26f0-0344-402d-be30-643eddb8b6a9 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:11.043952 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb26f0-0344-402d-be30-643eddb8b6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:11.115416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-32cb26f0-0344-402d-be30-643eddb8b6a9 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:11.116614 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-32cb26f0-0344-402d-be30-643eddb8b6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:11.134797 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:11.216051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 670/1345] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:11.499878 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571bc6ba-0bb5-4087-83a6-cf4cadd2a8e9 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:11.501441 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-571bc6ba-0bb5-4087-83a6-cf4cadd2a8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:11.605727 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571bc6ba-0bb5-4087-83a6-cf4cadd2a8e9 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:11.681013 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581c6ae-b0b2-42e3-b2dd-7651b3265518 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:11.681653 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581c6ae-b0b2-42e3-b2dd-7651b3265518 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:11.681965 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581c6ae-b0b2-42e3-b2dd-7651b3265518 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:11.682244 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581c6ae-b0b2-42e3-b2dd-7651b3265518 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:11.684918 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c581c6ae-b0b2-42e3-b2dd-7651b3265518 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:11.733762 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:11.804929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581c6ae-b0b2-42e3-b2dd-7651b3265518 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:11.806848 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581c6ae-b0b2-42e3-b2dd-7651b3265518 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:11.825434 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 676/1346] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:05:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:05:16.322811 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5ea39-4988-4809-a6c9-6a09797943ab None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:16.323310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5ea39-4988-4809-a6c9-6a09797943ab None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:16.323417 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5ea39-4988-4809-a6c9-6a09797943ab None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:16.324698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 671/1347] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:05:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:05:16.330224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c2457-4d75-48d7-908d-8a698a4a7c28 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:16.330668 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c2457-4d75-48d7-908d-8a698a4a7c28 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:16.330779 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c2457-4d75-48d7-908d-8a698a4a7c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:16.401294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-ce8c2457-4d75-48d7-908d-8a698a4a7c28 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:16.402570 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ce8c2457-4d75-48d7-908d-8a698a4a7c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:16.502246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:16.510023 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 677/1348] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:16.802037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5ea39-4988-4809-a6c9-6a09797943ab None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:16.803700 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-74a5ea39-4988-4809-a6c9-6a09797943ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:16.913311 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5ea39-4988-4809-a6c9-6a09797943ab None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:16.961390 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:16.987711 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ef0e1-4439-45cb-888e-3a41b5dc6036 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:16.988316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ef0e1-4439-45cb-888e-3a41b5dc6036 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:16.988594 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ef0e1-4439-45cb-888e-3a41b5dc6036 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:16.988863 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ef0e1-4439-45cb-888e-3a41b5dc6036 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:16.990513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-e75ef0e1-4439-45cb-888e-3a41b5dc6036 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:17.099138 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ef0e1-4439-45cb-888e-3a41b5dc6036 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:17.100559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ef0e1-4439-45cb-888e-3a41b5dc6036 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:17.118767 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 672/1349] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:05:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:05:22.016723 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ac7ca-8ea4-4e5d-b645-f6769da5e1ce None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:22.017231 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ac7ca-8ea4-4e5d-b645-f6769da5e1ce None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:22.017338 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ac7ca-8ea4-4e5d-b645-f6769da5e1ce None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:22.024982 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 678/1350] 10.222.0.73 () {54 vars in 921 bytes} [Mon Aug 29 20:05:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:22.032627 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5f6c5-730d-47d5-bee0-89e971694057 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:22.032627 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5f6c5-730d-47d5-bee0-89e971694057 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:22.032627 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5f6c5-730d-47d5-bee0-89e971694057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:22.147440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-6ce5f6c5-730d-47d5-bee0-89e971694057 None None] MFA Rules not processed for user `1ab2e318b2854c46bc7ed2151a44a2e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:22.149733 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-6ce5f6c5-730d-47d5-bee0-89e971694057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:22.171252 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:22.311907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 673/1351] 10.222.0.73 () {58 vars in 1018 bytes} [Mon Aug 29 20:05:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:26.616691 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879af0a5-6ed1-4be4-9ffb-5a1c0634cef8 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:26.617278 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879af0a5-6ed1-4be4-9ffb-5a1c0634cef8 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:26.617369 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879af0a5-6ed1-4be4-9ffb-5a1c0634cef8 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:26.618751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 679/1352] 10.222.0.73 () {54 vars in 921 bytes} [Mon Aug 29 20:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:26.624114 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab75b8-32ae-4fcf-ab47-8f8e13c0fc0b None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:26.624592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab75b8-32ae-4fcf-ab47-8f8e13c0fc0b None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:26.624704 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab75b8-32ae-4fcf-ab47-8f8e13c0fc0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:26.710877 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-78ab75b8-32ae-4fcf-ab47-8f8e13c0fc0b None None] MFA Rules not processed for user `1ab2e318b2854c46bc7ed2151a44a2e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:26.712805 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-78ab75b8-32ae-4fcf-ab47-8f8e13c0fc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:26.787427 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:26.827654 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 674/1353] 10.222.0.73 () {58 vars in 1018 bytes} [Mon Aug 29 20:05:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:28.279858 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99e677-c2d9-46c5-946f-ef2f0301158a None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:28.280411 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99e677-c2d9-46c5-946f-ef2f0301158a None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:28.280411 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99e677-c2d9-46c5-946f-ef2f0301158a None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:28.281762 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 680/1354] 10.222.0.73 () {54 vars in 921 bytes} [Mon Aug 29 20:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:28.286891 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca56ad0-e750-406a-9317-3561077cb0de None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:28.287342 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca56ad0-e750-406a-9317-3561077cb0de None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:28.287441 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca56ad0-e750-406a-9317-3561077cb0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:28.368681 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-9ca56ad0-e750-406a-9317-3561077cb0de None None] MFA Rules not processed for user `1ab2e318b2854c46bc7ed2151a44a2e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:28.369851 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9ca56ad0-e750-406a-9317-3561077cb0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:28.478576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 675/1355] 10.222.0.73 () {58 vars in 1018 bytes} [Mon Aug 29 20:05:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:31.799917 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d961a-dc83-4e09-8693-55e04aa79d60 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:31.800525 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d961a-dc83-4e09-8693-55e04aa79d60 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:31.800525 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d961a-dc83-4e09-8693-55e04aa79d60 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:31.801799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 681/1356] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:31.807503 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057cc32-0457-4b14-a3a6-00c592c6d148 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:31.807933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057cc32-0457-4b14-a3a6-00c592c6d148 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:31.808044 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057cc32-0457-4b14-a3a6-00c592c6d148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:31.844710 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:31.884627 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-d057cc32-0457-4b14-a3a6-00c592c6d148 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:31.886295 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d057cc32-0457-4b14-a3a6-00c592c6d148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:32.012102 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 676/1357] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:32.318119 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3855a-5f34-428f-ac1f-8725214bc9ac None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:32.318383 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3855a-5f34-428f-ac1f-8725214bc9ac None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:32.318980 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3855a-5f34-428f-ac1f-8725214bc9ac None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:32.320471 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 682/1358] 10.222.0.73 () {54 vars in 962 bytes} [Mon Aug 29 20:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:32.331859 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172cce2e-df0c-4baa-82aa-140db58f2b9c None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:32.332490 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172cce2e-df0c-4baa-82aa-140db58f2b9c None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:32.333046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172cce2e-df0c-4baa-82aa-140db58f2b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:32.448785 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-172cce2e-df0c-4baa-82aa-140db58f2b9c None None] MFA Rules not processed for user `95cf77a92a174ff8b0503baf5c7e63be`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:32.450099 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-172cce2e-df0c-4baa-82aa-140db58f2b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:32.503102 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:32.577325 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 677/1359] 10.222.0.73 () {58 vars in 1059 bytes} [Mon Aug 29 20:05:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:32.586280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a3855a-5f34-428f-ac1f-8725214bc9ac None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:32.587804 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a3a3855a-5f34-428f-ac1f-8725214bc9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:32.743928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a3855a-5f34-428f-ac1f-8725214bc9ac None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:32.827453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:32.875297 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a41bb6-92da-4856-8f46-20148dee0e02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:32.875910 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a41bb6-92da-4856-8f46-20148dee0e02 service nova] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:32.876189 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a41bb6-92da-4856-8f46-20148dee0e02 service nova] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:32.876459 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a41bb6-92da-4856-8f46-20148dee0e02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:32.878496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a5a41bb6-92da-4856-8f46-20148dee0e02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:32.983615 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a41bb6-92da-4856-8f46-20148dee0e02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:32.985208 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a41bb6-92da-4856-8f46-20148dee0e02 service nova] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:33.005539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 683/1360] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:05:35.177235 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd04ec-dd1c-4ec7-ac03-9d10c878d9a2 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:35.178383 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd04ec-dd1c-4ec7-ac03-9d10c878d9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:35.178383 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd04ec-dd1c-4ec7-ac03-9d10c878d9a2 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:35.186114 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 678/1361] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:05:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:35.189290 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5a020-6cca-4932-b5a9-ead273a53020 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:35.189947 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5a020-6cca-4932-b5a9-ead273a53020 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:35.189947 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5a020-6cca-4932-b5a9-ead273a53020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:35.277237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-14a5a020-6cca-4932-b5a9-ead273a53020 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:35.278513 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-14a5a020-6cca-4932-b5a9-ead273a53020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:35.320150 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:35.388767 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 684/1362] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:35.690503 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd04ec-dd1c-4ec7-ac03-9d10c878d9a2 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:35.692119 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-b1fd04ec-dd1c-4ec7-ac03-9d10c878d9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:35.861119 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd04ec-dd1c-4ec7-ac03-9d10c878d9a2 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:35.871648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:35.993162 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0a57d-a089-4436-8524-f4a8c0a8e711 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:35.993819 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0a57d-a089-4436-8524-f4a8c0a8e711 service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:35.994099 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0a57d-a089-4436-8524-f4a8c0a8e711 service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:35.994370 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0a57d-a089-4436-8524-f4a8c0a8e711 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:35.995992 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-49d0a57d-a089-4436-8524-f4a8c0a8e711 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:36.117045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d0a57d-a089-4436-8524-f4a8c0a8e711 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:36.118768 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d0a57d-a089-4436-8524-f4a8c0a8e711 service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:36.139700 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 679/1363] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:05:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:05:38.422374 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f0294-b1d2-4188-846d-5dd2d1f6600f None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:38.423209 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f0294-b1d2-4188-846d-5dd2d1f6600f None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:38.423487 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f0294-b1d2-4188-846d-5dd2d1f6600f None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:38.424855 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 685/1364] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:05:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:38.431003 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af089e6-888b-4671-b9f9-f46ecc066149 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:38.431560 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af089e6-888b-4671-b9f9-f46ecc066149 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:38.431560 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af089e6-888b-4671-b9f9-f46ecc066149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:38.448618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:38.503425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-8af089e6-888b-4671-b9f9-f46ecc066149 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:38.504656 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-8af089e6-888b-4671-b9f9-f46ecc066149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:38.609753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 680/1365] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:38.909071 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6ae6d-1e95-4cd4-8a54-a05fb686a1df None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:38.909740 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6ae6d-1e95-4cd4-8a54-a05fb686a1df None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:38.910088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6ae6d-1e95-4cd4-8a54-a05fb686a1df None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:38.911833 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 686/1366] 10.222.0.73 () {54 vars in 962 bytes} [Mon Aug 29 20:05:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:38.917638 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732314c-9e26-46e7-a0d0-0539d2eb769d None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:38.918075 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732314c-9e26-46e7-a0d0-0539d2eb769d None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:38.918443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732314c-9e26-46e7-a0d0-0539d2eb769d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:39.002867 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-f732314c-9e26-46e7-a0d0-0539d2eb769d None None] MFA Rules not processed for user `95cf77a92a174ff8b0503baf5c7e63be`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:39.004187 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-f732314c-9e26-46e7-a0d0-0539d2eb769d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:39.043503 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:39.140046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 681/1367] 10.222.0.73 () {58 vars in 1059 bytes} [Mon Aug 29 20:05:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:39.148431 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6ae6d-1e95-4cd4-8a54-a05fb686a1df None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:39.150132 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-6ec6ae6d-1e95-4cd4-8a54-a05fb686a1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:39.311608 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6ae6d-1e95-4cd4-8a54-a05fb686a1df None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:39.336261 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:39.444407 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d83a37-f262-4106-b33a-98d253efe6df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:39.445663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d83a37-f262-4106-b33a-98d253efe6df service nova] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:39.446010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d83a37-f262-4106-b33a-98d253efe6df service nova] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:39.446291 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d83a37-f262-4106-b33a-98d253efe6df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:39.447979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-16d83a37-f262-4106-b33a-98d253efe6df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:39.542605 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d83a37-f262-4106-b33a-98d253efe6df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:39.544664 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d83a37-f262-4106-b33a-98d253efe6df service nova] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:39.567204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 687/1368] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:05:40.177320 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e1603-f763-47cf-a448-04ae23d209a6 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:40.178039 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e1603-f763-47cf-a448-04ae23d209a6 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:40.178321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e1603-f763-47cf-a448-04ae23d209a6 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:40.179679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 682/1369] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:05:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:40.185098 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfddc37-ca5f-4376-a2f0-893c59ee5445 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:40.185491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfddc37-ca5f-4376-a2f0-893c59ee5445 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:40.185903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfddc37-ca5f-4376-a2f0-893c59ee5445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:40.219424 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:40.257006 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-5bfddc37-ca5f-4376-a2f0-893c59ee5445 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:40.258386 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-5bfddc37-ca5f-4376-a2f0-893c59ee5445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:40.361686 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 688/1370] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:40.925153 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df436e-eae7-43f7-adfa-b3d0a2f73c11 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:40.925551 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df436e-eae7-43f7-adfa-b3d0a2f73c11 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:40.926006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df436e-eae7-43f7-adfa-b3d0a2f73c11 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:40.928101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 683/1371] 10.222.0.73 () {54 vars in 964 bytes} [Mon Aug 29 20:05:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:40.933236 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72719d0-ffa7-4fe8-94c1-c718f4e7c3ce None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:40.933682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72719d0-ffa7-4fe8-94c1-c718f4e7c3ce None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:40.933889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72719d0-ffa7-4fe8-94c1-c718f4e7c3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:41.023257 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-c72719d0-ffa7-4fe8-94c1-c718f4e7c3ce None None] MFA Rules not processed for user `4a0b2ca8d14a47239a26912e58348f5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:41.024486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c72719d0-ffa7-4fe8-94c1-c718f4e7c3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:41.075171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:41.121458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 689/1372] 10.222.0.73 () {58 vars in 1061 bytes} [Mon Aug 29 20:05:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:41.129171 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df436e-eae7-43f7-adfa-b3d0a2f73c11 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:41.130858 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-e3df436e-eae7-43f7-adfa-b3d0a2f73c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:41.244835 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df436e-eae7-43f7-adfa-b3d0a2f73c11 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:41.296961 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:41.345565 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a415ae-8282-4ae9-b245-890c02a32428 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:41.346191 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a415ae-8282-4ae9-b245-890c02a32428 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:41.346515 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a415ae-8282-4ae9-b245-890c02a32428 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:41.346804 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a415ae-8282-4ae9-b245-890c02a32428 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:41.348338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c3a415ae-8282-4ae9-b245-890c02a32428 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:41.471763 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a415ae-8282-4ae9-b245-890c02a32428 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:41.473092 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a415ae-8282-4ae9-b245-890c02a32428 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:41.501693 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 684/1373] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:05:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:05:41.558226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db369e-c842-4cb2-9c2a-a1779bbec09d None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:41.558398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db369e-c842-4cb2-9c2a-a1779bbec09d None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:41.558860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db369e-c842-4cb2-9c2a-a1779bbec09d None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:41.562206 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 690/1374] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:41.565560 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31550f9f-9ce7-46dc-b4d4-1b1099b6a7e7 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:41.565848 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31550f9f-9ce7-46dc-b4d4-1b1099b6a7e7 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:41.566443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31550f9f-9ce7-46dc-b4d4-1b1099b6a7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:41.635182 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-31550f9f-9ce7-46dc-b4d4-1b1099b6a7e7 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:41.636415 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-31550f9f-9ce7-46dc-b4d4-1b1099b6a7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:41.659891 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:41.747107 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 685/1375] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:42.040972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db369e-c842-4cb2-9c2a-a1779bbec09d None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:42.042643 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f5db369e-c842-4cb2-9c2a-a1779bbec09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:42.194855 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db369e-c842-4cb2-9c2a-a1779bbec09d None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:42.260827 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:42.324770 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e659b-e3c4-4f20-8189-145b761e1562 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:42.325299 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e659b-e3c4-4f20-8189-145b761e1562 service nova] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:42.325578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e659b-e3c4-4f20-8189-145b761e1562 service nova] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:42.325878 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e659b-e3c4-4f20-8189-145b761e1562 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:42.327462 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-050e659b-e3c4-4f20-8189-145b761e1562 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:42.420885 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050e659b-e3c4-4f20-8189-145b761e1562 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:42.422541 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050e659b-e3c4-4f20-8189-145b761e1562 service nova] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:42.441287 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 691/1376] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:05:44.399959 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2033d-8bbe-46f4-8e0e-791a827bb1cc None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:44.400484 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2033d-8bbe-46f4-8e0e-791a827bb1cc None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:44.400599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2033d-8bbe-46f4-8e0e-791a827bb1cc None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:44.401883 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 686/1377] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:44.407010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01c7c6-be75-4eb2-b943-60fab82c8b23 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:44.407457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01c7c6-be75-4eb2-b943-60fab82c8b23 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:44.407564 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01c7c6-be75-4eb2-b943-60fab82c8b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:44.482031 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-fb01c7c6-be75-4eb2-b943-60fab82c8b23 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:44.483235 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-fb01c7c6-be75-4eb2-b943-60fab82c8b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:44.490774 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:44.613653 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 692/1378] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:44.913021 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2033d-8bbe-46f4-8e0e-791a827bb1cc None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:44.914583 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d0b2033d-8bbe-46f4-8e0e-791a827bb1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:45.114814 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2033d-8bbe-46f4-8e0e-791a827bb1cc None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:45.119513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:45.241814 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a1801-7824-4e5a-b199-0f582a150aca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:45.242353 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a1801-7824-4e5a-b199-0f582a150aca service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:45.242619 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a1801-7824-4e5a-b199-0f582a150aca service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:45.242886 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a1801-7824-4e5a-b199-0f582a150aca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:45.244490 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-120a1801-7824-4e5a-b199-0f582a150aca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:45.341200 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a1801-7824-4e5a-b199-0f582a150aca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:45.342816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a1801-7824-4e5a-b199-0f582a150aca service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:45.362219 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 687/1379] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:05:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:05:47.306298 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107d1ff-61e0-4d18-b271-444df4123fab None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:47.306835 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107d1ff-61e0-4d18-b271-444df4123fab None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:47.306835 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107d1ff-61e0-4d18-b271-444df4123fab None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:47.308373 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 693/1380] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:47.313249 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93753993-4339-4beb-8290-084c9874c443 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:47.313646 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93753993-4339-4beb-8290-084c9874c443 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:47.313865 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93753993-4339-4beb-8290-084c9874c443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:47.325560 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:47.386960 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-93753993-4339-4beb-8290-084c9874c443 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:47.388401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-93753993-4339-4beb-8290-084c9874c443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:47.499087 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 688/1381] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:47.789241 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107d1ff-61e0-4d18-b271-444df4123fab None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:47.790952 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-b107d1ff-61e0-4d18-b271-444df4123fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:47.950560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:47.962766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107d1ff-61e0-4d18-b271-444df4123fab None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:48.094251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a3e26-6323-472a-b00c-fee10cbc289c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:48.095057 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a3e26-6323-472a-b00c-fee10cbc289c service nova] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:48.095470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a3e26-6323-472a-b00c-fee10cbc289c service nova] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:48.095890 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a3e26-6323-472a-b00c-fee10cbc289c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:48.097666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-6c8a3e26-6323-472a-b00c-fee10cbc289c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:48.195603 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a3e26-6323-472a-b00c-fee10cbc289c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:48.197203 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a3e26-6323-472a-b00c-fee10cbc289c service nova] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:48.209486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:48.218843 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 694/1382] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:05:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:05:50.232776 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8117fa-5cb2-4a9e-b8a4-b093960fe303 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:50.233430 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8117fa-5cb2-4a9e-b8a4-b093960fe303 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:50.233430 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8117fa-5cb2-4a9e-b8a4-b093960fe303 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:50.234707 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 689/1383] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:50.240172 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6737b-bf01-4ffd-b5c2-ea1168f361b5 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:50.240727 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6737b-bf01-4ffd-b5c2-ea1168f361b5 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:50.240727 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6737b-bf01-4ffd-b5c2-ea1168f361b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:50.325040 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-b8e6737b-bf01-4ffd-b5c2-ea1168f361b5 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:50.326662 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-b8e6737b-bf01-4ffd-b5c2-ea1168f361b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:50.456218 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 695/1384] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:50.747675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8117fa-5cb2-4a9e-b8a4-b093960fe303 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:50.749253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-7b8117fa-5cb2-4a9e-b8a4-b093960fe303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:50.839917 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:50.905316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8117fa-5cb2-4a9e-b8a4-b093960fe303 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:51.024966 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d2725-ad56-43a4-b81a-c599b17ace65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:51.025448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d2725-ad56-43a4-b81a-c599b17ace65 service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:51.025756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d2725-ad56-43a4-b81a-c599b17ace65 service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:51.026091 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d2725-ad56-43a4-b81a-c599b17ace65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:51.027703 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-7c9d2725-ad56-43a4-b81a-c599b17ace65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:51.080151 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:51.122747 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d2725-ad56-43a4-b81a-c599b17ace65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:51.124334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d2725-ad56-43a4-b81a-c599b17ace65 service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:51.142641 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 690/1385] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:05:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:05:53.157361 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f48e32-4dd1-4d4f-9c4d-1899d1bb6b86 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:53.157928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f48e32-4dd1-4d4f-9c4d-1899d1bb6b86 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:53.158131 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f48e32-4dd1-4d4f-9c4d-1899d1bb6b86 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:53.159902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 696/1386] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:05:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:53.166098 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125b087-28f2-4052-8cdb-1e520b90fc68 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:53.166533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125b087-28f2-4052-8cdb-1e520b90fc68 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:53.166666 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125b087-28f2-4052-8cdb-1e520b90fc68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:53.247588 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-3125b087-28f2-4052-8cdb-1e520b90fc68 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:53.249157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-3125b087-28f2-4052-8cdb-1e520b90fc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:53.362777 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 691/1387] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:53.663590 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f48e32-4dd1-4d4f-9c4d-1899d1bb6b86 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:53.665183 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-47f48e32-4dd1-4d4f-9c4d-1899d1bb6b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:53.748343 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:53.823823 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f48e32-4dd1-4d4f-9c4d-1899d1bb6b86 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:53.946022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a1151-665c-4c90-8d98-b35285f4b2fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:53.946580 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a1151-665c-4c90-8d98-b35285f4b2fd service nova] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:53.946872 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a1151-665c-4c90-8d98-b35285f4b2fd service nova] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:53.947185 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a1151-665c-4c90-8d98-b35285f4b2fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:53.948843 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-806a1151-665c-4c90-8d98-b35285f4b2fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:53.989279 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:54.043472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a1151-665c-4c90-8d98-b35285f4b2fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:54.045080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a1151-665c-4c90-8d98-b35285f4b2fd service nova] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:54.064901 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 697/1388] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:05:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:05:56.077164 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6641fa5-7530-4d4f-b68a-6b5792687c70 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:56.077879 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6641fa5-7530-4d4f-b68a-6b5792687c70 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:56.077879 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6641fa5-7530-4d4f-b68a-6b5792687c70 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:56.079189 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 692/1389] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:05:56.085661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d54869-d43f-44a8-8e11-e5d8c71215fb None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:56.086607 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d54869-d43f-44a8-8e11-e5d8c71215fb None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:56.087407 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d54869-d43f-44a8-8e11-e5d8c71215fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:56.154811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-41d54869-d43f-44a8-8e11-e5d8c71215fb None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:56.155993 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-41d54869-d43f-44a8-8e11-e5d8c71215fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:56.255300 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 698/1390] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:56.551819 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6641fa5-7530-4d4f-b68a-6b5792687c70 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:56.553369 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d6641fa5-7530-4d4f-b68a-6b5792687c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:56.587461 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:56.706185 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6641fa5-7530-4d4f-b68a-6b5792687c70 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:56.809555 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:56.821376 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03003194-c038-4f6a-aaec-03c7c693c029 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:56.822013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03003194-c038-4f6a-aaec-03c7c693c029 service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:56.822294 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03003194-c038-4f6a-aaec-03c7c693c029 service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:56.822565 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03003194-c038-4f6a-aaec-03c7c693c029 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:56.824208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-03003194-c038-4f6a-aaec-03c7c693c029 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:56.917059 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03003194-c038-4f6a-aaec-03c7c693c029 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:56.918797 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03003194-c038-4f6a-aaec-03c7c693c029 service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:56.920568 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f172b00-7ac6-4061-8bde-0fd9d41dda01 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:56.920860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f172b00-7ac6-4061-8bde-0fd9d41dda01 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:56.921157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f172b00-7ac6-4061-8bde-0fd9d41dda01 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:56.922570 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 699/1391] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:56.927635 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf721c2f-4622-48d8-9455-318c9f1008df None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:56.927919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf721c2f-4622-48d8-9455-318c9f1008df None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:56.928196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf721c2f-4622-48d8-9455-318c9f1008df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:56.938084 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 693/1392] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:05:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:05:56.944385 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:57.012780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-bf721c2f-4622-48d8-9455-318c9f1008df None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:57.012780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-bf721c2f-4622-48d8-9455-318c9f1008df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:57.113532 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 700/1393] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:05:57.416089 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953ba37-2b87-4bc6-9211-c06bb4893a89 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:57.416577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953ba37-2b87-4bc6-9211-c06bb4893a89 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:57.416698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953ba37-2b87-4bc6-9211-c06bb4893a89 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:57.418066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 694/1394] 10.222.0.73 () {54 vars in 967 bytes} [Mon Aug 29 20:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:57.423168 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e020b-1477-4b1d-bffb-4e42733e2b3d None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:57.423456 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e020b-1477-4b1d-bffb-4e42733e2b3d None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:57.423731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e020b-1477-4b1d-bffb-4e42733e2b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:57.500100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-020e020b-1477-4b1d-bffb-4e42733e2b3d None None] MFA Rules not processed for user `103b64b4b172467692d02851b2955b81`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:57.501305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-020e020b-1477-4b1d-bffb-4e42733e2b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:57.536442 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:57.579470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 701/1395] 10.222.0.73 () {58 vars in 1064 bytes} [Mon Aug 29 20:05:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:57.587517 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953ba37-2b87-4bc6-9211-c06bb4893a89 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:57.589077 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9953ba37-2b87-4bc6-9211-c06bb4893a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:57.694969 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953ba37-2b87-4bc6-9211-c06bb4893a89 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:57.708133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:57.771663 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34816214-c3bc-4cc4-ace3-bfb1dfd07d2a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b64b4b172467692d02851b2955b81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:57.772269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34816214-c3bc-4cc4-ace3-bfb1dfd07d2a service glance] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:57.772546 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34816214-c3bc-4cc4-ace3-bfb1dfd07d2a service glance] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:57.772816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34816214-c3bc-4cc4-ace3-bfb1dfd07d2a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:57.774468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-34816214-c3bc-4cc4-ace3-bfb1dfd07d2a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:57.894049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34816214-c3bc-4cc4-ace3-bfb1dfd07d2a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:57.895481 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34816214-c3bc-4cc4-ace3-bfb1dfd07d2a service glance] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:57.914372 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 695/1396] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:05:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:05:58.657402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c753643-d943-486b-8dd1-2a372b422c47 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:58.657922 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c753643-d943-486b-8dd1-2a372b422c47 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:58.658223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c753643-d943-486b-8dd1-2a372b422c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:58.725982 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-8c753643-d943-486b-8dd1-2a372b422c47 None None] MFA Rules not processed for user `e0c76478b6694782b896cdf9cda32d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:58.727195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-8c753643-d943-486b-8dd1-2a372b422c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:58.817544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 702/1397] 10.222.0.73 () {58 vars in 1009 bytes} [Mon Aug 29 20:05:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:58.865671 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df0b5a-cc1c-44c5-8664-5ca063164761 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:58.866225 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df0b5a-cc1c-44c5-8664-5ca063164761 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:58.866332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df0b5a-cc1c-44c5-8664-5ca063164761 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:58.867620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 696/1398] 10.222.0.73 () {54 vars in 943 bytes} [Mon Aug 29 20:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:58.874370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebf107-2605-402e-bd58-dc83482e89d1 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:58.874659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebf107-2605-402e-bd58-dc83482e89d1 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:58.874932 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebf107-2605-402e-bd58-dc83482e89d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:58.915704 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ab913-9234-43a9-b357-9e0c8fc1f4b6 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:58.915908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ab913-9234-43a9-b357-9e0c8fc1f4b6 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:58.916163 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ab913-9234-43a9-b357-9e0c8fc1f4b6 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:58.917509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 697/1399] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:05:58.922470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f869ad-3d55-48ab-857d-dabfcb58bc36 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:58.922758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f869ad-3d55-48ab-857d-dabfcb58bc36 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:58.923039 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f869ad-3d55-48ab-857d-dabfcb58bc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:58.936220 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:58.955425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-30ebf107-2605-402e-bd58-dc83482e89d1 None None] MFA Rules not processed for user `b92e30da8f394f3abc19b9e86ef126e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:58.956669 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-30ebf107-2605-402e-bd58-dc83482e89d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:58.976883 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:58.992179 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-76f869ad-3d55-48ab-857d-dabfcb58bc36 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:58.993407 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-76f869ad-3d55-48ab-857d-dabfcb58bc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:59.066145 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 703/1400] 10.222.0.73 () {58 vars in 1040 bytes} [Mon Aug 29 20:05:58 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:05:59.076172 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ebf107-2605-402e-bd58-dc83482e89d1 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:59.086945 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ebf107-2605-402e-bd58-dc83482e89d1 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:59.102780 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 698/1401] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:05:59.193197 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:59.198854 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbaea37-9cc4-499c-b1dd-73b177c23619 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92e30da8f394f3abc19b9e86ef126e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:59.199468 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaea37-9cc4-499c-b1dd-73b177c23619 service swift] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:59.199753 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaea37-9cc4-499c-b1dd-73b177c23619 service swift] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:59.200042 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaea37-9cc4-499c-b1dd-73b177c23619 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:59.201654 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-9dbaea37-9cc4-499c-b1dd-73b177c23619 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:59.293012 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbaea37-9cc4-499c-b1dd-73b177c23619 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:59.294645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbaea37-9cc4-499c-b1dd-73b177c23619 service swift] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:59.303615 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 704/1402] 10.222.0.73 () {60 vars in 1384 bytes} [Mon Aug 29 20:05:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 20:05:59.400638 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f869ad-3d55-48ab-857d-dabfcb58bc36 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:59.401901 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-76f869ad-3d55-48ab-857d-dabfcb58bc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:59.537333 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f869ad-3d55-48ab-857d-dabfcb58bc36 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:59.552082 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:59.664255 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708ea8c-44ff-4a73-8611-140a05d2796b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:59.664872 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708ea8c-44ff-4a73-8611-140a05d2796b service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:59.665157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708ea8c-44ff-4a73-8611-140a05d2796b service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:59.665433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708ea8c-44ff-4a73-8611-140a05d2796b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:59.667101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-5708ea8c-44ff-4a73-8611-140a05d2796b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:59.771603 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5708ea8c-44ff-4a73-8611-140a05d2796b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:59.773475 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5708ea8c-44ff-4a73-8611-140a05d2796b service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:59.794691 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 699/1403] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:05:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:06:01.797605 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd771d1f-c3ad-4759-9fee-886ed1397792 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:01.798204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd771d1f-c3ad-4759-9fee-886ed1397792 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:01.798204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd771d1f-c3ad-4759-9fee-886ed1397792 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:01.799588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 705/1404] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:01.805800 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb99a0-6db2-424f-bc0d-8e8d69aaa376 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:01.806290 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb99a0-6db2-424f-bc0d-8e8d69aaa376 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:01.806556 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb99a0-6db2-424f-bc0d-8e8d69aaa376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:01.828062 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:01.882179 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-a6fb99a0-6db2-424f-bc0d-8e8d69aaa376 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:01.883797 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-a6fb99a0-6db2-424f-bc0d-8e8d69aaa376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:01.997742 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 700/1405] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:06:02.290081 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd771d1f-c3ad-4759-9fee-886ed1397792 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:02.291615 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-fd771d1f-c3ad-4759-9fee-886ed1397792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:02.460342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:02.471691 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd771d1f-c3ad-4759-9fee-886ed1397792 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:02.602044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce8675a-8b2e-4109-8d77-8384542880f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:02.602758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8675a-8b2e-4109-8d77-8384542880f4 service nova] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:02.603111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8675a-8b2e-4109-8d77-8384542880f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:02.603450 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8675a-8b2e-4109-8d77-8384542880f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:02.605066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-4ce8675a-8b2e-4109-8d77-8384542880f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:02.710220 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce8675a-8b2e-4109-8d77-8384542880f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:02.711971 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce8675a-8b2e-4109-8d77-8384542880f4 service nova] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:02.723401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:02.733159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 706/1406] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:06:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:06:04.704547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d216aab-8af7-4ad2-b015-1d54a04a9ada None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:04.705111 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d216aab-8af7-4ad2-b015-1d54a04a9ada None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:04.705111 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d216aab-8af7-4ad2-b015-1d54a04a9ada None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:04.713363 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f8076-4b5d-4f00-a6dd-e6125c84ecca None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:04.713962 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f8076-4b5d-4f00-a6dd-e6125c84ecca None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:04.713962 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f8076-4b5d-4f00-a6dd-e6125c84ecca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:04.718569 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 701/1407] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:06:04 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:04.789773 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-9f4f8076-4b5d-4f00-a6dd-e6125c84ecca None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:04.790985 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-9f4f8076-4b5d-4f00-a6dd-e6125c84ecca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:04.899974 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 707/1408] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:06:05.201349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d216aab-8af7-4ad2-b015-1d54a04a9ada None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:05.204339 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-7d216aab-8af7-4ad2-b015-1d54a04a9ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:05.328405 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:05.393426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d216aab-8af7-4ad2-b015-1d54a04a9ada None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:05.527860 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9410b-1418-4768-8131-d96c02d5fa9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:05.528631 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9410b-1418-4768-8131-d96c02d5fa9a service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:05.528974 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9410b-1418-4768-8131-d96c02d5fa9a service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:05.529295 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9410b-1418-4768-8131-d96c02d5fa9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:05.531462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-8bd9410b-1418-4768-8131-d96c02d5fa9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:05.604700 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:05.638538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9410b-1418-4768-8131-d96c02d5fa9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:05.640119 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9410b-1418-4768-8131-d96c02d5fa9a service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:05.662810 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 702/1409] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:06:14.897867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=67 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:15.659145 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=20 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:21.497599 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e7b0d-132e-4894-a880-f96d1f6f1a3f None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:21.498165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e7b0d-132e-4894-a880-f96d1f6f1a3f None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:21.498165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e7b0d-132e-4894-a880-f96d1f6f1a3f None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:21.499439 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 708/1410] 10.222.0.73 () {54 vars in 918 bytes} [Mon Aug 29 20:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:21.505055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa80a5a-5f4d-4dd6-b032-97ee9e88e115 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:21.505487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa80a5a-5f4d-4dd6-b032-97ee9e88e115 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:21.505584 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa80a5a-5f4d-4dd6-b032-97ee9e88e115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:21.579421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-baa80a5a-5f4d-4dd6-b032-97ee9e88e115 None None] MFA Rules not processed for user `1ab2e318b2854c46bc7ed2151a44a2e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:21.580617 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-baa80a5a-5f4d-4dd6-b032-97ee9e88e115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:21.685903 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 703/1411] 10.222.0.73 () {58 vars in 1015 bytes} [Mon Aug 29 20:06:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:06:21.700741 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66cdb4-6113-424e-b795-df8ddfec469a None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:21.701028 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66cdb4-6113-424e-b795-df8ddfec469a None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:21.701304 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66cdb4-6113-424e-b795-df8ddfec469a None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:21.702653 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 709/1412] 10.222.0.73 () {54 vars in 918 bytes} [Mon Aug 29 20:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:21.707473 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f6248-5c19-426c-828f-9a9651a8232c None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:21.707757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f6248-5c19-426c-828f-9a9651a8232c None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:21.708030 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f6248-5c19-426c-828f-9a9651a8232c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:21.779860 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:21.784211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-3d3f6248-5c19-426c-828f-9a9651a8232c None None] MFA Rules not processed for user `95cf77a92a174ff8b0503baf5c7e63be`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:21.785785 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-3d3f6248-5c19-426c-828f-9a9651a8232c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:21.900343 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 704/1413] 10.222.0.73 () {58 vars in 1015 bytes} [Mon Aug 29 20:06:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:06:24.281274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d9d19-1f07-48fc-808f-d60e881d4041 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:24.281820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d9d19-1f07-48fc-808f-d60e881d4041 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:24.281955 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d9d19-1f07-48fc-808f-d60e881d4041 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:24.283275 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:24.288502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3664b8e-2a09-44b4-adac-a2427ea62f02 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:24.288930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3664b8e-2a09-44b4-adac-a2427ea62f02 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:24.289070 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3664b8e-2a09-44b4-adac-a2427ea62f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:24.356457 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-c3664b8e-2a09-44b4-adac-a2427ea62f02 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:24.357737 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c3664b8e-2a09-44b4-adac-a2427ea62f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:24.412535 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:24.456145 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:06:24.753850 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d9d19-1f07-48fc-808f-d60e881d4041 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:24.755341 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-db6d9d19-1f07-48fc-808f-d60e881d4041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:24.911117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d9d19-1f07-48fc-808f-d60e881d4041 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:24.987870 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:25.027412 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4511a955-cba7-4c36-9df8-c787668833ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:25.028012 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4511a955-cba7-4c36-9df8-c787668833ec service nova] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:25.028289 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4511a955-cba7-4c36-9df8-c787668833ec service nova] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:25.028556 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4511a955-cba7-4c36-9df8-c787668833ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:25.030216 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-4511a955-cba7-4c36-9df8-c787668833ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:25.121584 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4511a955-cba7-4c36-9df8-c787668833ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:25.123304 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4511a955-cba7-4c36-9df8-c787668833ec service nova] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:25.141155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 711/1416] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:06:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:06:26.045140 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819c245-1e81-4bec-b0e0-b07bd9cafb0c None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:26.045684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819c245-1e81-4bec-b0e0-b07bd9cafb0c None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:26.045684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819c245-1e81-4bec-b0e0-b07bd9cafb0c None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:26.047063 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:26.052585 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e873d1-ccc3-421a-97fe-814f5044c362 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:26.053015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e873d1-ccc3-421a-97fe-814f5044c362 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:26.053156 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e873d1-ccc3-421a-97fe-814f5044c362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:26.130052 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-33e873d1-ccc3-421a-97fe-814f5044c362 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:26.131262 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-33e873d1-ccc3-421a-97fe-814f5044c362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:26.158059 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:26.227495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:06:26.377834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819c245-1e81-4bec-b0e0-b07bd9cafb0c None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:26.379424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-8819c245-1e81-4bec-b0e0-b07bd9cafb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:26.524158 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819c245-1e81-4bec-b0e0-b07bd9cafb0c None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:26.563872 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:26.635432 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f0784d-d539-4298-9430-739f5f0d75fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:26.635949 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f0784d-d539-4298-9430-739f5f0d75fa service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:26.636226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f0784d-d539-4298-9430-739f5f0d75fa service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:26.636493 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f0784d-d539-4298-9430-739f5f0d75fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:26.638136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-23f0784d-d539-4298-9430-739f5f0d75fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:26.727070 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f0784d-d539-4298-9430-739f5f0d75fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:26.728646 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f0784d-d539-4298-9430-739f5f0d75fa service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:26.746783 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 707/1419] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:06:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:06:27.625471 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd087971-dc2a-4989-b2da-953c00144a2e None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:27.626797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd087971-dc2a-4989-b2da-953c00144a2e None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:27.626797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd087971-dc2a-4989-b2da-953c00144a2e None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:27.628021 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:06:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:27.632885 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0fb1c-af94-4c91-8de8-083f460c29a9 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:27.633288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0fb1c-af94-4c91-8de8-083f460c29a9 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:27.633431 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0fb1c-af94-4c91-8de8-083f460c29a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:27.687792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:27.711499 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-8dd0fb1c-af94-4c91-8de8-083f460c29a9 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:27.712717 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-8dd0fb1c-af94-4c91-8de8-083f460c29a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:27.808655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:06:27.964481 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd087971-dc2a-4989-b2da-953c00144a2e None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:27.966140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-fd087971-dc2a-4989-b2da-953c00144a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:28.115772 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd087971-dc2a-4989-b2da-953c00144a2e None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:28.128116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:28.229735 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b90af-a19a-430c-8dd1-53f7c09475a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:28.230256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b90af-a19a-430c-8dd1-53f7c09475a2 service nova] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:28.230535 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b90af-a19a-430c-8dd1-53f7c09475a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:28.230808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b90af-a19a-430c-8dd1-53f7c09475a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:28.232407 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-245b90af-a19a-430c-8dd1-53f7c09475a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:28.321312 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b90af-a19a-430c-8dd1-53f7c09475a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:28.322937 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b90af-a19a-430c-8dd1-53f7c09475a2 service nova] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:28.341272 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 714/1422] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:06:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:06:29.236129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c235cb-cabf-417b-b017-d582cff1d20b None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:29.236772 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c235cb-cabf-417b-b017-d582cff1d20b None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:29.236772 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c235cb-cabf-417b-b017-d582cff1d20b None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:29.238083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:06:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:29.243296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd136a4-d951-4f36-a712-aaafe607fb72 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:29.243732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd136a4-d951-4f36-a712-aaafe607fb72 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:29.243873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd136a4-d951-4f36-a712-aaafe607fb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:29.265486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:29.325957 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-edd136a4-d951-4f36-a712-aaafe607fb72 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:29.327157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-edd136a4-d951-4f36-a712-aaafe607fb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:29.429523 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:06:29.587281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c235cb-cabf-417b-b017-d582cff1d20b None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:29.589323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-96c235cb-cabf-417b-b017-d582cff1d20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:29.704523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:29.746859 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c235cb-cabf-417b-b017-d582cff1d20b None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:29.867455 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd268c5-3679-4f43-9097-629cf6c0a537 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:29.868142 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd268c5-3679-4f43-9097-629cf6c0a537 service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:29.868427 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd268c5-3679-4f43-9097-629cf6c0a537 service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:29.868754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd268c5-3679-4f43-9097-629cf6c0a537 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:29.870582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-1bd268c5-3679-4f43-9097-629cf6c0a537 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:29.949596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:29.964171 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd268c5-3679-4f43-9097-629cf6c0a537 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:29.965693 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd268c5-3679-4f43-9097-629cf6c0a537 service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:29.984211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 710/1425] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:06:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:06:30.859010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f13b20-704f-485d-a1c3-0dedbcc25ff7 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:30.859573 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f13b20-704f-485d-a1c3-0dedbcc25ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:30.859573 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f13b20-704f-485d-a1c3-0dedbcc25ff7 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:30.861027 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:30.866108 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4fc1b2-c4f4-4123-a361-a5a2cb17d85e None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:30.866538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4fc1b2-c4f4-4123-a361-a5a2cb17d85e None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:30.866649 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4fc1b2-c4f4-4123-a361-a5a2cb17d85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:30.944998 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-3d4fc1b2-c4f4-4123-a361-a5a2cb17d85e None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:30.946197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-3d4fc1b2-c4f4-4123-a361-a5a2cb17d85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:31.044106 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:06:31.200287 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f13b20-704f-485d-a1c3-0dedbcc25ff7 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:31.201780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-19f13b20-704f-485d-a1c3-0dedbcc25ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:31.291838 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:31.360301 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f13b20-704f-485d-a1c3-0dedbcc25ff7 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:31.487306 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd2f07-fa89-4cfa-9a88-9a1da9413af5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:31.487816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd2f07-fa89-4cfa-9a88-9a1da9413af5 service nova] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:31.488089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd2f07-fa89-4cfa-9a88-9a1da9413af5 service nova] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:31.488473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd2f07-fa89-4cfa-9a88-9a1da9413af5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:31.489999 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-01fd2f07-fa89-4cfa-9a88-9a1da9413af5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:31.552862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:31.606387 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd2f07-fa89-4cfa-9a88-9a1da9413af5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:31.608003 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd2f07-fa89-4cfa-9a88-9a1da9413af5 service nova] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:31.627957 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 717/1428] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:06:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:06:32.494468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6f6e4-9bc3-4bf8-b4de-5bfc4edb0b7a None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:32.495031 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6f6e4-9bc3-4bf8-b4de-5bfc4edb0b7a None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:32.495031 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6f6e4-9bc3-4bf8-b4de-5bfc4edb0b7a None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:32.496311 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:32.501758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd62e2-88b0-4cc0-ad83-e4777af4585d None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:32.502227 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd62e2-88b0-4cc0-ad83-e4777af4585d None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:32.502350 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd62e2-88b0-4cc0-ad83-e4777af4585d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:32.581317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-2ccd62e2-88b0-4cc0-ad83-e4777af4585d None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:32.582589 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-2ccd62e2-88b0-4cc0-ad83-e4777af4585d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:32.677040 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:06:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:06:32.830358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d6f6e4-9bc3-4bf8-b4de-5bfc4edb0b7a None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:32.831919 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-f2d6f6e4-9bc3-4bf8-b4de-5bfc4edb0b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:32.881659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:32.981300 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d6f6e4-9bc3-4bf8-b4de-5bfc4edb0b7a None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:33.097169 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c229241-3c44-48cc-b415-1845f44d29e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:33.097813 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c229241-3c44-48cc-b415-1845f44d29e6 service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:33.098101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c229241-3c44-48cc-b415-1845f44d29e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:33.098377 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c229241-3c44-48cc-b415-1845f44d29e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:33.100044 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-2c229241-3c44-48cc-b415-1845f44d29e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:33.107209 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:33.191154 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c229241-3c44-48cc-b415-1845f44d29e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:33.192720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c229241-3c44-48cc-b415-1845f44d29e6 service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:33.210151 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 713/1431] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:06:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:06:34.033873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e5219-6aa9-42b3-a496-69bea480d1cc None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:34.034436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e5219-6aa9-42b3-a496-69bea480d1cc None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:34.034436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e5219-6aa9-42b3-a496-69bea480d1cc None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:34.035750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:34.040835 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b55bc5-96ab-4ef3-8162-ca0b51ca261d None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:34.041291 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b55bc5-96ab-4ef3-8162-ca0b51ca261d None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:34.041465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b55bc5-96ab-4ef3-8162-ca0b51ca261d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:34.119549 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-b4b55bc5-96ab-4ef3-8162-ca0b51ca261d None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:34.120758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-b4b55bc5-96ab-4ef3-8162-ca0b51ca261d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:34.187315 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:34.217019 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:06:34.372037 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e5219-6aa9-42b3-a496-69bea480d1cc None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:34.373484 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-083e5219-6aa9-42b3-a496-69bea480d1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:34.391915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:34.524208 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e5219-6aa9-42b3-a496-69bea480d1cc None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:34.612016 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:34.758273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e605c-6aa9-4ec7-8c7c-df51f7f68aa0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:34.758872 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e605c-6aa9-4ec7-8c7c-df51f7f68aa0 service nova] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:34.759147 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e605c-6aa9-4ec7-8c7c-df51f7f68aa0 service nova] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:34.759412 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e605c-6aa9-4ec7-8c7c-df51f7f68aa0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:34.761000 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-998e605c-6aa9-4ec7-8c7c-df51f7f68aa0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:34.852592 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998e605c-6aa9-4ec7-8c7c-df51f7f68aa0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:34.854255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998e605c-6aa9-4ec7-8c7c-df51f7f68aa0 service nova] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:34.872872 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 720/1434] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:06:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:06:35.682175 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c7be5-5240-4ee2-8a23-d90abda302db None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:35.682743 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c7be5-5240-4ee2-8a23-d90abda302db None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:35.682743 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c7be5-5240-4ee2-8a23-d90abda302db None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:35.684104 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:35.689670 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f720f0-c267-49a1-916d-57d1c7c9e3e5 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:35.690168 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f720f0-c267-49a1-916d-57d1c7c9e3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:35.690301 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f720f0-c267-49a1-916d-57d1c7c9e3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:35.765600 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-20f720f0-c267-49a1-916d-57d1c7c9e3e5 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:35.767816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-20f720f0-c267-49a1-916d-57d1c7c9e3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:35.815685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:35.872789 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:06:36.030265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c7be5-5240-4ee2-8a23-d90abda302db None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:36.031750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-388c7be5-5240-4ee2-8a23-d90abda302db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:36.179481 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c7be5-5240-4ee2-8a23-d90abda302db None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:36.235906 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:36.294993 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7acd0-ee2c-4754-bbcc-ca5504d5f7ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:36.295527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7acd0-ee2c-4754-bbcc-ca5504d5f7ed service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:36.295806 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7acd0-ee2c-4754-bbcc-ca5504d5f7ed service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:36.296079 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7acd0-ee2c-4754-bbcc-ca5504d5f7ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:36.297847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c0d7acd0-ee2c-4754-bbcc-ca5504d5f7ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:36.390115 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7acd0-ee2c-4754-bbcc-ca5504d5f7ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:36.392502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7acd0-ee2c-4754-bbcc-ca5504d5f7ed service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:36.411297 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 716/1437] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:06:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:06:37.225737 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac9e66-349e-4ecf-bc7e-348b4647a1bc None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:37.226289 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac9e66-349e-4ecf-bc7e-348b4647a1bc None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:37.226289 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac9e66-349e-4ecf-bc7e-348b4647a1bc None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:37.227617 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:37.232498 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc129b-49c4-451e-8210-cf888e0e7f38 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:37.232928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc129b-49c4-451e-8210-cf888e0e7f38 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:37.233034 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc129b-49c4-451e-8210-cf888e0e7f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:37.309788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-8ccc129b-49c4-451e-8210-cf888e0e7f38 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:37.311000 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-8ccc129b-49c4-451e-8210-cf888e0e7f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:37.313759 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:37.407293 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:06:37.557593 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac9e66-349e-4ecf-bc7e-348b4647a1bc None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:37.559138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f0ac9e66-349e-4ecf-bc7e-348b4647a1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:37.706127 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac9e66-349e-4ecf-bc7e-348b4647a1bc None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:37.732631 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:37.823484 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54267d81-dabe-42af-b2a4-cfc743b3ab3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:37.824204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54267d81-dabe-42af-b2a4-cfc743b3ab3e service nova] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:37.824485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54267d81-dabe-42af-b2a4-cfc743b3ab3e service nova] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:37.824752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54267d81-dabe-42af-b2a4-cfc743b3ab3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:37.826385 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-54267d81-dabe-42af-b2a4-cfc743b3ab3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:37.916888 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54267d81-dabe-42af-b2a4-cfc743b3ab3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:37.918509 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54267d81-dabe-42af-b2a4-cfc743b3ab3e service nova] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:37.936888 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 723/1440] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:06:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:06:39.753456 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e197ac8-07e8-4eb9-91a7-e7fc7fa5fbcb None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:39.754082 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e197ac8-07e8-4eb9-91a7-e7fc7fa5fbcb None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:39.754082 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e197ac8-07e8-4eb9-91a7-e7fc7fa5fbcb None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:39.755376 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:39.760679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6e82d-194f-4ac2-8b04-ab2b4093e4fd None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:39.761135 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6e82d-194f-4ac2-8b04-ab2b4093e4fd None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:39.761234 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6e82d-194f-4ac2-8b04-ab2b4093e4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:39.803771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:39.829570 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-bcb6e82d-194f-4ac2-8b04-ab2b4093e4fd None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:39.830797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-bcb6e82d-194f-4ac2-8b04-ab2b4093e4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:39.931316 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:06:40.220348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e197ac8-07e8-4eb9-91a7-e7fc7fa5fbcb None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:40.221964 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-3e197ac8-07e8-4eb9-91a7-e7fc7fa5fbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:40.324036 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e197ac8-07e8-4eb9-91a7-e7fc7fa5fbcb None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:40.356741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:40.398440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ec298-b9c3-49df-b46a-cae32eba01b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:40.399041 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ec298-b9c3-49df-b46a-cae32eba01b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:40.399317 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ec298-b9c3-49df-b46a-cae32eba01b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:40.399583 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ec298-b9c3-49df-b46a-cae32eba01b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:40.401168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-1c3ec298-b9c3-49df-b46a-cae32eba01b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:40.508207 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ec298-b9c3-49df-b46a-cae32eba01b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:40.509638 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ec298-b9c3-49df-b46a-cae32eba01b0 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:40.528702 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 719/1443] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:06:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:06:42.589970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a789db-ddb0-4102-b41f-0ab2e25462e8 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:42.590517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a789db-ddb0-4102-b41f-0ab2e25462e8 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:42.590517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a789db-ddb0-4102-b41f-0ab2e25462e8 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:42.591852 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:42.597375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da7118-2869-448f-820b-5f53f74fcb39 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:42.597841 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da7118-2869-448f-820b-5f53f74fcb39 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:42.598017 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da7118-2869-448f-820b-5f53f74fcb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:42.665257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-23da7118-2869-448f-820b-5f53f74fcb39 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:42.666434 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-23da7118-2869-448f-820b-5f53f74fcb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:42.680039 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:42.885406 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:06:43.175341 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a789db-ddb0-4102-b41f-0ab2e25462e8 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:43.176894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-78a789db-ddb0-4102-b41f-0ab2e25462e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:43.282726 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a789db-ddb0-4102-b41f-0ab2e25462e8 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:43.294955 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:43.358165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4dba29-0d92-4b3b-8b70-6fea5a25e898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:43.358678 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dba29-0d92-4b3b-8b70-6fea5a25e898 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:43.358990 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dba29-0d92-4b3b-8b70-6fea5a25e898 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:43.359256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dba29-0d92-4b3b-8b70-6fea5a25e898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:43.360822 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-bc4dba29-0d92-4b3b-8b70-6fea5a25e898 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:43.467510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4dba29-0d92-4b3b-8b70-6fea5a25e898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:43.469590 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4dba29-0d92-4b3b-8b70-6fea5a25e898 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:43.487920 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 726/1446] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:06:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:06:45.689387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b326b7-4cb9-4fcb-924e-1d8f60928a82 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:45.689952 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b326b7-4cb9-4fcb-924e-1d8f60928a82 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:45.689952 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b326b7-4cb9-4fcb-924e-1d8f60928a82 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:45.691290 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:45.696467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922caea-e6bc-4277-962a-eefecc451bb5 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:45.696919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922caea-e6bc-4277-962a-eefecc451bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:45.697028 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922caea-e6bc-4277-962a-eefecc451bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:45.744846 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:45.763820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-a922caea-e6bc-4277-962a-eefecc451bb5 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:45.765908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a922caea-e6bc-4277-962a-eefecc451bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:45.861855 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:06:46.149049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b326b7-4cb9-4fcb-924e-1d8f60928a82 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:46.150600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d3b326b7-4cb9-4fcb-924e-1d8f60928a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:46.252262 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b326b7-4cb9-4fcb-924e-1d8f60928a82 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:46.292634 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:46.326486 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f450ff-6bef-4a28-bdc8-b366f50f42cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:46.327132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f450ff-6bef-4a28-bdc8-b366f50f42cf service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:46.327946 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f450ff-6bef-4a28-bdc8-b366f50f42cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:46.328293 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f450ff-6bef-4a28-bdc8-b366f50f42cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:46.329923 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-76f450ff-6bef-4a28-bdc8-b366f50f42cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:46.440505 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f450ff-6bef-4a28-bdc8-b366f50f42cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:46.441858 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f450ff-6bef-4a28-bdc8-b366f50f42cf service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:46.460081 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 722/1449] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:06:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:06:48.519953 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7553c-c18e-4916-a882-a60c35331fa6 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:48.520475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7553c-c18e-4916-a882-a60c35331fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:48.520475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7553c-c18e-4916-a882-a60c35331fa6 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:48.521820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:48.527009 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21605251-6f7e-4a75-8f64-bba61e34f4e2 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:48.527414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21605251-6f7e-4a75-8f64-bba61e34f4e2 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:48.527557 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21605251-6f7e-4a75-8f64-bba61e34f4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:48.591947 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-21605251-6f7e-4a75-8f64-bba61e34f4e2 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:48.593682 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-21605251-6f7e-4a75-8f64-bba61e34f4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:48.613836 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:48.693777 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:06:48.986209 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7553c-c18e-4916-a882-a60c35331fa6 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:48.987761 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-e8b7553c-c18e-4916-a882-a60c35331fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:49.090877 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7553c-c18e-4916-a882-a60c35331fa6 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:49.111407 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:49.165726 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada96e9a-b138-4118-94c6-193a09bb2710 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:49.166351 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada96e9a-b138-4118-94c6-193a09bb2710 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:49.166634 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada96e9a-b138-4118-94c6-193a09bb2710 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:49.166906 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada96e9a-b138-4118-94c6-193a09bb2710 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:49.168504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ada96e9a-b138-4118-94c6-193a09bb2710 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:49.281104 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada96e9a-b138-4118-94c6-193a09bb2710 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:49.282492 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada96e9a-b138-4118-94c6-193a09bb2710 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:49.300715 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 729/1452] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:06:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:06:51.279739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb04d09-dd85-4b37-969c-e45a020a5c75 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:51.280518 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb04d09-dd85-4b37-969c-e45a020a5c75 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:51.280518 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb04d09-dd85-4b37-969c-e45a020a5c75 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:51.281645 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:51.286849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcff95c-dd55-4433-b65c-971a88f177e2 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:51.287287 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcff95c-dd55-4433-b65c-971a88f177e2 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:51.287409 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcff95c-dd55-4433-b65c-971a88f177e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:51.351743 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:51.355239 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-5fcff95c-dd55-4433-b65c-971a88f177e2 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:51.357053 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-5fcff95c-dd55-4433-b65c-971a88f177e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:51.454943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:06:51.738484 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb04d09-dd85-4b37-969c-e45a020a5c75 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:51.740062 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-dbb04d09-dd85-4b37-969c-e45a020a5c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:51.847099 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb04d09-dd85-4b37-969c-e45a020a5c75 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:51.896375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:51.921726 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29af5f-106e-4547-af98-c63a85dc48aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:51.922333 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29af5f-106e-4547-af98-c63a85dc48aa service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:51.922605 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29af5f-106e-4547-af98-c63a85dc48aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:51.922874 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29af5f-106e-4547-af98-c63a85dc48aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:51.924480 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-ab29af5f-106e-4547-af98-c63a85dc48aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:52.032836 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29af5f-106e-4547-af98-c63a85dc48aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:52.034210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29af5f-106e-4547-af98-c63a85dc48aa service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:52.052382 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 725/1455] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:06:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:06:54.009212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c837de-e09e-4598-a5be-11fb81c0a2b2 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:54.009813 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c837de-e09e-4598-a5be-11fb81c0a2b2 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:54.009813 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c837de-e09e-4598-a5be-11fb81c0a2b2 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:54.011125 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:54.017512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79f398-31db-43fc-8ba8-15f710644627 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:54.018331 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79f398-31db-43fc-8ba8-15f710644627 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:54.018331 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79f398-31db-43fc-8ba8-15f710644627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:54.083895 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-5b79f398-31db-43fc-8ba8-15f710644627 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:54.085160 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-5b79f398-31db-43fc-8ba8-15f710644627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:54.114546 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:54.182548 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:06:54.468137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c837de-e09e-4598-a5be-11fb81c0a2b2 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:54.469625 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c2c837de-e09e-4598-a5be-11fb81c0a2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:54.572405 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c837de-e09e-4598-a5be-11fb81c0a2b2 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:54.599759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:54.646595 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1002ea0-a2ce-43a9-92ae-ab876c4ad75a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:54.647198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1002ea0-a2ce-43a9-92ae-ab876c4ad75a service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:54.647476 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1002ea0-a2ce-43a9-92ae-ab876c4ad75a service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:54.647747 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1002ea0-a2ce-43a9-92ae-ab876c4ad75a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:54.649341 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f1002ea0-a2ce-43a9-92ae-ab876c4ad75a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:54.758128 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1002ea0-a2ce-43a9-92ae-ab876c4ad75a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:54.759558 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1002ea0-a2ce-43a9-92ae-ab876c4ad75a service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:54.778432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 732/1458] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:06:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:06:56.746029 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadd827-864d-4e11-9cb0-b16c453db2b3 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:56.746537 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadd827-864d-4e11-9cb0-b16c453db2b3 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:56.746651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadd827-864d-4e11-9cb0-b16c453db2b3 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:56.747924 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:06:56.753553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d6c1c-5648-4696-b227-3f20c2e7a72c None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:56.754014 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d6c1c-5648-4696-b227-3f20c2e7a72c None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:56.754142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d6c1c-5648-4696-b227-3f20c2e7a72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:56.821384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-107d6c1c-5648-4696-b227-3f20c2e7a72c None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:56.822630 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-107d6c1c-5648-4696-b227-3f20c2e7a72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:56.831562 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:56.920438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:06:57.207380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadd827-864d-4e11-9cb0-b16c453db2b3 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:57.208921 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9fadd827-864d-4e11-9cb0-b16c453db2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:57.327763 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadd827-864d-4e11-9cb0-b16c453db2b3 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:57.381561 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:57.400374 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825a923-1757-49d5-a6da-59c40d9ac7da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:57.400975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825a923-1757-49d5-a6da-59c40d9ac7da service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:57.401253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825a923-1757-49d5-a6da-59c40d9ac7da service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:57.401533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825a923-1757-49d5-a6da-59c40d9ac7da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:57.403166 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-7825a923-1757-49d5-a6da-59c40d9ac7da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:57.511950 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825a923-1757-49d5-a6da-59c40d9ac7da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:57.513366 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825a923-1757-49d5-a6da-59c40d9ac7da service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:57.531432 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 728/1461] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:06:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:06:59.483488 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8d784-b1b7-4c0a-8911-d4596fe29853 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:59.484024 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8d784-b1b7-4c0a-8911-d4596fe29853 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:59.484024 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8d784-b1b7-4c0a-8911-d4596fe29853 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:59.485453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:06:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:59.490809 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67b0f1-598d-4c91-b67c-1373b27003f7 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:59.491223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67b0f1-598d-4c91-b67c-1373b27003f7 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:59.491361 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67b0f1-598d-4c91-b67c-1373b27003f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:59.556496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-cb67b0f1-598d-4c91-b67c-1373b27003f7 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:59.557692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-cb67b0f1-598d-4c91-b67c-1373b27003f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:59.595755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:59.656067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:06:59.938437 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8d784-b1b7-4c0a-8911-d4596fe29853 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:59.939988 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-79a8d784-b1b7-4c0a-8911-d4596fe29853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:00.044802 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8d784-b1b7-4c0a-8911-d4596fe29853 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:00.085176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:00.122565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648639b-b7fa-44f1-93f1-83a032fdd989 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:00.123177 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648639b-b7fa-44f1-93f1-83a032fdd989 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:00.124165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648639b-b7fa-44f1-93f1-83a032fdd989 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:00.124440 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648639b-b7fa-44f1-93f1-83a032fdd989 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:00.126079 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-6648639b-b7fa-44f1-93f1-83a032fdd989 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:00.234660 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648639b-b7fa-44f1-93f1-83a032fdd989 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:00.236027 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648639b-b7fa-44f1-93f1-83a032fdd989 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:00.255982 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 735/1464] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:06:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:07:02.168856 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7a210-cd75-41f6-b122-dab55b2131f9 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:02.169453 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7a210-cd75-41f6-b122-dab55b2131f9 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:02.169453 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7a210-cd75-41f6-b122-dab55b2131f9 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:02.171148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:02.176495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370740e-5440-4c85-b046-b56c1f002888 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:02.176936 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370740e-5440-4c85-b046-b56c1f002888 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:02.177040 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370740e-5440-4c85-b046-b56c1f002888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:02.242385 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-c370740e-5440-4c85-b046-b56c1f002888 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:02.243681 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c370740e-5440-4c85-b046-b56c1f002888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:02.261539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:02.339502 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:07:02.621941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7a210-cd75-41f6-b122-dab55b2131f9 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:02.624003 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-2db7a210-cd75-41f6-b122-dab55b2131f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:02.727610 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7a210-cd75-41f6-b122-dab55b2131f9 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:02.795514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:02.802755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e288d-7da2-4858-b514-f40d36ccc45b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:02.803393 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e288d-7da2-4858-b514-f40d36ccc45b service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:02.803669 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e288d-7da2-4858-b514-f40d36ccc45b service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:02.803959 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e288d-7da2-4858-b514-f40d36ccc45b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:02.805578 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-296e288d-7da2-4858-b514-f40d36ccc45b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:02.912773 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e288d-7da2-4858-b514-f40d36ccc45b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:02.914225 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e288d-7da2-4858-b514-f40d36ccc45b service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:02.932848 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 731/1467] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:07:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:07:04.866519 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435d80a-ba93-4fce-9a38-65b76eaa1c52 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:04.867046 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435d80a-ba93-4fce-9a38-65b76eaa1c52 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:04.867168 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435d80a-ba93-4fce-9a38-65b76eaa1c52 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:04.868665 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:04.874314 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf95d8b-2709-4e12-bbce-0fbed31e70b4 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:04.874765 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf95d8b-2709-4e12-bbce-0fbed31e70b4 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:04.874878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf95d8b-2709-4e12-bbce-0fbed31e70b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:04.940936 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-1bf95d8b-2709-4e12-bbce-0fbed31e70b4 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:04.942203 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-1bf95d8b-2709-4e12-bbce-0fbed31e70b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:04.991014 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:05.041467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:07:05.340927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435d80a-ba93-4fce-9a38-65b76eaa1c52 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:05.342685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-5435d80a-ba93-4fce-9a38-65b76eaa1c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:05.455625 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435d80a-ba93-4fce-9a38-65b76eaa1c52 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:05.503383 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:05.530230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a343d-666a-4bf2-b3cd-98d2a50667c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:05.530759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a343d-666a-4bf2-b3cd-98d2a50667c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:05.531031 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a343d-666a-4bf2-b3cd-98d2a50667c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:05.531295 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a343d-666a-4bf2-b3cd-98d2a50667c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:05.532872 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f49a343d-666a-4bf2-b3cd-98d2a50667c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:05.639573 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a343d-666a-4bf2-b3cd-98d2a50667c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:05.641348 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a343d-666a-4bf2-b3cd-98d2a50667c5 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:05.659639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 738/1470] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:07:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:07:15.039718 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=22 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:15.657305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=66 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:25.367168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba337db8-9f7c-4933-91c2-171e679a2632 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:25.367697 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba337db8-9f7c-4933-91c2-171e679a2632 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:25.367697 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba337db8-9f7c-4933-91c2-171e679a2632 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:25.368978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 733/1471] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:25.374382 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8545f-5ecf-454e-9578-c07ca72dfcd9 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:25.374803 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8545f-5ecf-454e-9578-c07ca72dfcd9 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:25.374934 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8545f-5ecf-454e-9578-c07ca72dfcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:25.442345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-5eb8545f-5ecf-454e-9578-c07ca72dfcd9 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:25.443555 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-5eb8545f-5ecf-454e-9578-c07ca72dfcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:25.540553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 739/1472] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:07:25.838699 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf935e-a482-44e3-9c2e-2379822c54b7 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:25.838982 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf935e-a482-44e3-9c2e-2379822c54b7 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:25.839259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf935e-a482-44e3-9c2e-2379822c54b7 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:25.840647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 734/1473] 10.222.0.73 () {54 vars in 967 bytes} [Mon Aug 29 20:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:25.845542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4a069-8741-4b5d-b37d-035763e6480b None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:25.845873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4a069-8741-4b5d-b37d-035763e6480b None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:25.846154 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4a069-8741-4b5d-b37d-035763e6480b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:25.923052 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-6ff4a069-8741-4b5d-b37d-035763e6480b None None] MFA Rules not processed for user `103b64b4b172467692d02851b2955b81`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:25.924272 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-6ff4a069-8741-4b5d-b37d-035763e6480b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:25.945187 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:26.005860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 740/1474] 10.222.0.73 () {58 vars in 1064 bytes} [Mon Aug 29 20:07:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:07:26.014320 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf935e-a482-44e3-9c2e-2379822c54b7 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:26.016125 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-5ecf935e-a482-44e3-9c2e-2379822c54b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:26.120630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf935e-a482-44e3-9c2e-2379822c54b7 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:26.194599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a6157-aea7-419c-b6fb-e3b972918aa9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b64b4b172467692d02851b2955b81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:26.195213 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a6157-aea7-419c-b6fb-e3b972918aa9 service glance] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:26.195496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a6157-aea7-419c-b6fb-e3b972918aa9 service glance] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:26.195790 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a6157-aea7-419c-b6fb-e3b972918aa9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:26.197463 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-7f5a6157-aea7-419c-b6fb-e3b972918aa9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:26.252371 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:26.310071 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a6157-aea7-419c-b6fb-e3b972918aa9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:26.311504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a6157-aea7-419c-b6fb-e3b972918aa9 service glance] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:26.329807 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 735/1475] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:07:28.364883 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adb81b-b67e-41ce-8b0e-983736108c93 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:28.365414 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adb81b-b67e-41ce-8b0e-983736108c93 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:28.365414 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adb81b-b67e-41ce-8b0e-983736108c93 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:28.366848 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 741/1476] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:28.372876 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883ba32-0b48-46a0-bb7b-ad230bac4ad2 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:28.373326 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883ba32-0b48-46a0-bb7b-ad230bac4ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:28.373438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883ba32-0b48-46a0-bb7b-ad230bac4ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:28.450976 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-7883ba32-0b48-46a0-bb7b-ad230bac4ad2 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:28.452204 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-7883ba32-0b48-46a0-bb7b-ad230bac4ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:28.549306 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 736/1477] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:07:28.830095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adb81b-b67e-41ce-8b0e-983736108c93 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:28.831649 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-d4adb81b-b67e-41ce-8b0e-983736108c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:28.957617 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adb81b-b67e-41ce-8b0e-983736108c93 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:29.007158 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:29.052547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeae6b0-8c73-4112-932c-bb0970e8e271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:29.053146 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeae6b0-8c73-4112-932c-bb0970e8e271 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:29.053423 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeae6b0-8c73-4112-932c-bb0970e8e271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:29.053693 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeae6b0-8c73-4112-932c-bb0970e8e271 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:29.054366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeae6b0-8c73-4112-932c-bb0970e8e271 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:29.056077 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: WARNING py.warnings [None req-beeae6b0-8c73-4112-932c-bb0970e8e271 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:07:29.056077 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: warnings.warn(msg) Aug 29 20:07:29.056077 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: Aug 29 20:07:29.057204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeae6b0-8c73-4112-932c-bb0970e8e271 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:29.076313 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 742/1478] 10.222.0.73 () {56 vars in 1108 bytes} [Mon Aug 29 20:07:28 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:07:31.018037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453fbc2-bb26-438d-b547-a18b84b0e15e None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:31.018550 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453fbc2-bb26-438d-b547-a18b84b0e15e None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:31.018666 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453fbc2-bb26-438d-b547-a18b84b0e15e None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:31.020042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 737/1479] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:31.025812 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411c3ce-b611-47aa-89e2-186b0e26dbad None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:31.026247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411c3ce-b611-47aa-89e2-186b0e26dbad None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:31.026364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411c3ce-b611-47aa-89e2-186b0e26dbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:31.092863 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-3411c3ce-b611-47aa-89e2-186b0e26dbad None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:31.094156 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-3411c3ce-b611-47aa-89e2-186b0e26dbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:31.196638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 743/1480] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:07:31.564565 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453fbc2-bb26-438d-b547-a18b84b0e15e None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:31.566175 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-0453fbc2-bb26-438d-b547-a18b84b0e15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:31.577020 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:31.735648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453fbc2-bb26-438d-b547-a18b84b0e15e None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:31.838972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:31.884848 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479d899-f3b8-48f1-bff1-2037fecde1d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:31.885434 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479d899-f3b8-48f1-bff1-2037fecde1d8 service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:31.885746 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479d899-f3b8-48f1-bff1-2037fecde1d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:31.886030 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479d899-f3b8-48f1-bff1-2037fecde1d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:31.888317 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-2479d899-f3b8-48f1-bff1-2037fecde1d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:31.998682 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479d899-f3b8-48f1-bff1-2037fecde1d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:32.000393 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479d899-f3b8-48f1-bff1-2037fecde1d8 service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:32.019083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 738/1481] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:07:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:07:34.141611 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e5ca6-bbaf-49dd-8950-55e016553a63 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:34.142208 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e5ca6-bbaf-49dd-8950-55e016553a63 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:34.142208 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e5ca6-bbaf-49dd-8950-55e016553a63 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:34.143528 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 744/1482] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:34.148716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e4f65-4d4e-4506-b581-be058a7589ab None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:34.149214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e4f65-4d4e-4506-b581-be058a7589ab None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:34.149326 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e4f65-4d4e-4506-b581-be058a7589ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:34.214067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-064e4f65-4d4e-4506-b581-be058a7589ab None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:34.215286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-064e4f65-4d4e-4506-b581-be058a7589ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:34.275255 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:34.324896 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 739/1483] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:07:34.618926 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e5ca6-bbaf-49dd-8950-55e016553a63 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:34.620467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c92e5ca6-bbaf-49dd-8950-55e016553a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:34.644104 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:34.724217 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e5ca6-bbaf-49dd-8950-55e016553a63 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:34.798346 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c510ebf-6a1c-48a0-862f-70cb8f1004b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:34.798953 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c510ebf-6a1c-48a0-862f-70cb8f1004b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:34.799231 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c510ebf-6a1c-48a0-862f-70cb8f1004b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:34.799503 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c510ebf-6a1c-48a0-862f-70cb8f1004b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:34.801089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-2c510ebf-6a1c-48a0-862f-70cb8f1004b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:34.877513 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:34.913562 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c510ebf-6a1c-48a0-862f-70cb8f1004b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:34.915042 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c510ebf-6a1c-48a0-862f-70cb8f1004b5 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:34.934960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 745/1484] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:07:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:07:36.922738 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244f611-bf83-45bf-9f5b-25615fbd07fd None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:36.923375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244f611-bf83-45bf-9f5b-25615fbd07fd None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:36.923375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244f611-bf83-45bf-9f5b-25615fbd07fd None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:36.924704 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 740/1485] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:36.930205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f079cf0-36eb-4002-8bfd-44eefd7fe35d None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:36.930660 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f079cf0-36eb-4002-8bfd-44eefd7fe35d None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:36.930800 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f079cf0-36eb-4002-8bfd-44eefd7fe35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:36.997365 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-2f079cf0-36eb-4002-8bfd-44eefd7fe35d None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:36.998524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-2f079cf0-36eb-4002-8bfd-44eefd7fe35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:37.099596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 746/1486] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:07:37.387896 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244f611-bf83-45bf-9f5b-25615fbd07fd None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:37.389964 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-4244f611-bf83-45bf-9f5b-25615fbd07fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:37.494259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244f611-bf83-45bf-9f5b-25615fbd07fd None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:37.565191 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:37.566897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf5c83-d8c0-4129-a33e-7d975f311baa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:37.567532 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf5c83-d8c0-4129-a33e-7d975f311baa service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:37.567978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf5c83-d8c0-4129-a33e-7d975f311baa service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:37.568511 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf5c83-d8c0-4129-a33e-7d975f311baa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:37.570581 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-57bf5c83-d8c0-4129-a33e-7d975f311baa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:37.678257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf5c83-d8c0-4129-a33e-7d975f311baa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:37.679599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf5c83-d8c0-4129-a33e-7d975f311baa service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:37.698521 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 741/1487] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:07:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:07:39.836129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e523f24-eb3d-41b7-9182-97e4ae814ac4 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:39.836706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e523f24-eb3d-41b7-9182-97e4ae814ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:39.836706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e523f24-eb3d-41b7-9182-97e4ae814ac4 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:39.838033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 747/1488] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:39.843397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e7c10-6c50-40db-b57a-309c02fb7181 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:39.843940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e7c10-6c50-40db-b57a-309c02fb7181 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:39.844046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e7c10-6c50-40db-b57a-309c02fb7181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:39.911362 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-407e7c10-6c50-40db-b57a-309c02fb7181 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:39.912578 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-407e7c10-6c50-40db-b57a-309c02fb7181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:39.965485 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:40.009610 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 742/1489] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:07:40.312447 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e523f24-eb3d-41b7-9182-97e4ae814ac4 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:40.314062 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-6e523f24-eb3d-41b7-9182-97e4ae814ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:40.350069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:40.417546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e523f24-eb3d-41b7-9182-97e4ae814ac4 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:40.490278 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6891db7-7c6b-41e3-8a47-1623d539019d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:40.490871 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6891db7-7c6b-41e3-8a47-1623d539019d service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:40.491152 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6891db7-7c6b-41e3-8a47-1623d539019d service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:40.491420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6891db7-7c6b-41e3-8a47-1623d539019d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:40.493155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c6891db7-7c6b-41e3-8a47-1623d539019d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:40.574583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:40.604743 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6891db7-7c6b-41e3-8a47-1623d539019d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:40.606683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6891db7-7c6b-41e3-8a47-1623d539019d service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:40.625601 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 748/1490] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:07:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:07:42.717678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc7b1a-e840-44ab-9c28-99101dcccb75 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:42.718295 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc7b1a-e840-44ab-9c28-99101dcccb75 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:42.718295 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc7b1a-e840-44ab-9c28-99101dcccb75 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:42.719598 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 743/1491] 10.222.0.73 () {54 vars in 927 bytes} [Mon Aug 29 20:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:42.724973 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358eed7b-84fe-4d91-991e-bc41dcd155b3 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:42.725407 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358eed7b-84fe-4d91-991e-bc41dcd155b3 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:42.725530 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358eed7b-84fe-4d91-991e-bc41dcd155b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:42.795063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-358eed7b-84fe-4d91-991e-bc41dcd155b3 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:42.796359 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-358eed7b-84fe-4d91-991e-bc41dcd155b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:42.894320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 749/1492] 10.222.0.73 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:07:43.193940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc7b1a-e840-44ab-9c28-99101dcccb75 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:43.195505 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-10fc7b1a-e840-44ab-9c28-99101dcccb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:43.301000 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc7b1a-e840-44ab-9c28-99101dcccb75 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:43.375217 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7c660-e2f9-4644-bebb-8fbef1056285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:43.375748 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7c660-e2f9-4644-bebb-8fbef1056285 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:43.376025 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7c660-e2f9-4644-bebb-8fbef1056285 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:43.376295 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7c660-e2f9-4644-bebb-8fbef1056285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:43.377940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-70a7c660-e2f9-4644-bebb-8fbef1056285 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:43.387529 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:43.487828 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a7c660-e2f9-4644-bebb-8fbef1056285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:43.489259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a7c660-e2f9-4644-bebb-8fbef1056285 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:43.507509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 744/1493] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:07:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:07:52.892470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=82 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:53.505286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=50 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:08:56.031263 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd06f1-7956-4fb9-8cfe-0b4dcf26a40c None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:08:56.031886 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd06f1-7956-4fb9-8cfe-0b4dcf26a40c None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:08:56.031886 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd06f1-7956-4fb9-8cfe-0b4dcf26a40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:08:56.107890 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-c9bd06f1-7956-4fb9-8cfe-0b4dcf26a40c None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:08:56.109098 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c9bd06f1-7956-4fb9-8cfe-0b4dcf26a40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:08:56.204740 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 750/1494] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:08:56.212762 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c2b9a-7302-4010-b9b7-6fcc3dcba418 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:08:56.213246 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c2b9a-7302-4010-b9b7-6fcc3dcba418 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:08:56.213362 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c2b9a-7302-4010-b9b7-6fcc3dcba418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:08:56.289188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-ab0c2b9a-7302-4010-b9b7-6fcc3dcba418 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:08:56.291196 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-ab0c2b9a-7302-4010-b9b7-6fcc3dcba418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:08:56.385632 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 745/1495] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:08:56.390534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd06f1-7956-4fb9-8cfe-0b4dcf26a40c None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:08:56.392041 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c9bd06f1-7956-4fb9-8cfe-0b4dcf26a40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:08:56.404206 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd06f1-7956-4fb9-8cfe-0b4dcf26a40c None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:08:56.464978 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:08:56.499596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9730437-9ec8-4258-aa10-d7a7a7de72bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:08:56.500199 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9730437-9ec8-4258-aa10-d7a7a7de72bc admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:08:56.500476 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9730437-9ec8-4258-aa10-d7a7a7de72bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:08:56.500746 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9730437-9ec8-4258-aa10-d7a7a7de72bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:08:56.501376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9730437-9ec8-4258-aa10-d7a7a7de72bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:08:56.501676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9730437-9ec8-4258-aa10-d7a7a7de72bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:08:56.502720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: WARNING py.warnings [None req-f9730437-9ec8-4258-aa10-d7a7a7de72bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:08:56.502720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: warnings.warn(msg) Aug 29 20:08:56.502720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: Aug 29 20:08:56.503589 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9730437-9ec8-4258-aa10-d7a7a7de72bc admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:08:56.515018 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 751/1496] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:08:56.523423 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c2b9a-7302-4010-b9b7-6fcc3dcba418 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:08:56.533201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c2b9a-7302-4010-b9b7-6fcc3dcba418 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:08:56.594803 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:08:56.629721 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e8c82-5856-40f9-8002-179022a69bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:08:56.630337 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e8c82-5856-40f9-8002-179022a69bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:08:56.630615 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e8c82-5856-40f9-8002-179022a69bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:08:56.630884 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e8c82-5856-40f9-8002-179022a69bd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:08:56.631502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e8c82-5856-40f9-8002-179022a69bd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:08:56.633259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: WARNING py.warnings [None req-815e8c82-5856-40f9-8002-179022a69bd7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:08:56.633259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: warnings.warn(msg) Aug 29 20:08:56.633259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: Aug 29 20:08:56.635212 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e8c82-5856-40f9-8002-179022a69bd7 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:08:56.690889 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 746/1497] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:08:56 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:08:56.695155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9730437-9ec8-4258-aa10-d7a7a7de72bc admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:08:56.696825 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f9730437-9ec8-4258-aa10-d7a7a7de72bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:08:56.730350 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9730437-9ec8-4258-aa10-d7a7a7de72bc admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:08:56.825895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a86743-3c80-403d-8bcd-8798f556652e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:08:56.826490 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a86743-3c80-403d-8bcd-8798f556652e admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:08:56.826763 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a86743-3c80-403d-8bcd-8798f556652e admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:08:56.827028 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a86743-3c80-403d-8bcd-8798f556652e admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:08:56.827655 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a86743-3c80-403d-8bcd-8798f556652e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:08:56.828651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: WARNING py.warnings [None req-44a86743-3c80-403d-8bcd-8798f556652e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:08:56.828651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: warnings.warn(msg) Aug 29 20:08:56.828651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: Aug 29 20:08:56.829878 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a86743-3c80-403d-8bcd-8798f556652e admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:08:56.871228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 752/1498] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:08:56 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:08:56.875237 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e8c82-5856-40f9-8002-179022a69bd7 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:08:56.876761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-815e8c82-5856-40f9-8002-179022a69bd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:08:56.937804 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e8c82-5856-40f9-8002-179022a69bd7 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:08:57.024502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:08:57.036031 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf4c94-9a6d-4498-9d1b-0e7d980fe905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:08:57.036639 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf4c94-9a6d-4498-9d1b-0e7d980fe905 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:08:57.036920 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf4c94-9a6d-4498-9d1b-0e7d980fe905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:08:57.037196 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf4c94-9a6d-4498-9d1b-0e7d980fe905 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:08:57.037873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf4c94-9a6d-4498-9d1b-0e7d980fe905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:08:57.038880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: WARNING py.warnings [None req-daaf4c94-9a6d-4498-9d1b-0e7d980fe905 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:08:57.038880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: warnings.warn(msg) Aug 29 20:08:57.038880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: Aug 29 20:08:57.039831 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf4c94-9a6d-4498-9d1b-0e7d980fe905 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:08:57.052133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 747/1499] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:08:57.056867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a86743-3c80-403d-8bcd-8798f556652e admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:08:57.058425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-44a86743-3c80-403d-8bcd-8798f556652e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:08:57.069799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a86743-3c80-403d-8bcd-8798f556652e admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:08:57.096462 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:08:57.171795 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607b6dd-4d01-447c-8a61-733126349d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:08:57.172296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607b6dd-4d01-447c-8a61-733126349d14 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:08:57.172576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607b6dd-4d01-447c-8a61-733126349d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:08:57.172851 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607b6dd-4d01-447c-8a61-733126349d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:08:57.173496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2607b6dd-4d01-447c-8a61-733126349d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:08:57.174546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: WARNING py.warnings [None req-2607b6dd-4d01-447c-8a61-733126349d14 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:08:57.174546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: warnings.warn(msg) Aug 29 20:08:57.174546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: Aug 29 20:08:57.175430 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2607b6dd-4d01-447c-8a61-733126349d14 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:08:57.189471 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 753/1500] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:08:57.193798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf4c94-9a6d-4498-9d1b-0e7d980fe905 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:08:57.195349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-daaf4c94-9a6d-4498-9d1b-0e7d980fe905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:08:57.229050 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf4c94-9a6d-4498-9d1b-0e7d980fe905 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:08:57.327848 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dce71c-c37e-4b27-a99f-0eeaad919c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:08:57.328528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dce71c-c37e-4b27-a99f-0eeaad919c7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:08:57.328789 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dce71c-c37e-4b27-a99f-0eeaad919c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:08:57.329061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dce71c-c37e-4b27-a99f-0eeaad919c7b admin admin] PATH_INFO: `/v3/projects/d03f551c5a9d47a29672411a463cb047/users/4575999ff7d64756bef3b338edce2779/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:08:57.351891 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dce71c-c37e-4b27-a99f-0eeaad919c7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d03f551c5a9d47a29672411a463cb047, user_id=4575999ff7d64756bef3b338edce2779, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:08:57.353809 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: WARNING py.warnings [None req-45dce71c-c37e-4b27-a99f-0eeaad919c7b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:08:57.353809 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: warnings.warn(msg) Aug 29 20:08:57.353809 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: Aug 29 20:08:57.355726 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dce71c-c37e-4b27-a99f-0eeaad919c7b admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:08:57.379799 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 748/1501] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:08:57 2022] PUT /identity/v3/projects/d03f551c5a9d47a29672411a463cb047/users/4575999ff7d64756bef3b338edce2779/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:08:57.392633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca7d3f-eed5-4e3a-ab1f-66610959ec31 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:08:57.392913 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca7d3f-eed5-4e3a-ab1f-66610959ec31 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:08:57.393181 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca7d3f-eed5-4e3a-ab1f-66610959ec31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:08:57.451344 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-daca7d3f-eed5-4e3a-ab1f-66610959ec31 None None] MFA Rules not processed for user `4575999ff7d64756bef3b338edce2779`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:08:57.452463 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-daca7d3f-eed5-4e3a-ab1f-66610959ec31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:08:57.533529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 754/1502] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:08:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:08:57.547596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a40283-2e7b-4bca-b03e-9643cdaf5bc9 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:08:57.547925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a40283-2e7b-4bca-b03e-9643cdaf5bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:08:57.548249 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a40283-2e7b-4bca-b03e-9643cdaf5bc9 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:08:57.549683 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 749/1503] 10.222.0.73 () {54 vars in 964 bytes} [Mon Aug 29 20:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:08:57.555335 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eb063-44a6-4d0d-849f-8495c2d0f718 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:08:57.555619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eb063-44a6-4d0d-849f-8495c2d0f718 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:08:57.555890 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eb063-44a6-4d0d-849f-8495c2d0f718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:08:57.590118 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:08:57.636240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-c48eb063-44a6-4d0d-849f-8495c2d0f718 None None] MFA Rules not processed for user `4a0b2ca8d14a47239a26912e58348f5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:08:57.637436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c48eb063-44a6-4d0d-849f-8495c2d0f718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:08:57.714934 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 755/1504] 10.222.0.73 () {58 vars in 1061 bytes} [Mon Aug 29 20:08:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:08:57.722798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a40283-2e7b-4bca-b03e-9643cdaf5bc9 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:08:57.724316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-52a40283-2e7b-4bca-b03e-9643cdaf5bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:08:57.775786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:08:57.831451 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a40283-2e7b-4bca-b03e-9643cdaf5bc9 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:08:57.907285 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4072819-050b-487a-a3f3-b269686ea8be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:08:57.907898 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4072819-050b-487a-a3f3-b269686ea8be service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:08:57.908177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4072819-050b-487a-a3f3-b269686ea8be service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:08:57.908448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4072819-050b-487a-a3f3-b269686ea8be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:08:57.910104 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-e4072819-050b-487a-a3f3-b269686ea8be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:08:58.010968 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4072819-050b-487a-a3f3-b269686ea8be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:08:58.012304 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4072819-050b-487a-a3f3-b269686ea8be service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:08:58.016051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:08:58.031722 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 750/1505] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:08:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:08:58.076947 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ef663-4a79-4b3c-b0b2-afd3d6cbb707 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:08:58.077366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ef663-4a79-4b3c-b0b2-afd3d6cbb707 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:08:58.077488 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ef663-4a79-4b3c-b0b2-afd3d6cbb707 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:08:58.078832 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 756/1506] 10.222.0.73 () {52 vars in 830 bytes} [Mon Aug 29 20:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:08:58.082439 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4072819-050b-487a-a3f3-b269686ea8be service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:08:58.083669 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-e4072819-050b-487a-a3f3-b269686ea8be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:08:58.185140 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4072819-050b-487a-a3f3-b269686ea8be service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:08:58.278892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0323c-fca9-4746-bf18-5f6e4109f520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:08:58.279639 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0323c-fca9-4746-bf18-5f6e4109f520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:08:58.279736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0323c-fca9-4746-bf18-5f6e4109f520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:08:58.279944 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0323c-fca9-4746-bf18-5f6e4109f520 admin admin] PATH_INFO: `/v3/users/4575999ff7d64756bef3b338edce2779` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:08:58.292416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0323c-fca9-4746-bf18-5f6e4109f520 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4575999ff7d64756bef3b338edce2779)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:08:58.293439 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: WARNING py.warnings [None req-4ca0323c-fca9-4746-bf18-5f6e4109f520 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:08:58.293439 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: warnings.warn(msg) Aug 29 20:08:58.293439 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: Aug 29 20:08:58.294682 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0323c-fca9-4746-bf18-5f6e4109f520 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:08:58.304285 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:08:58.362860 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-4ca0323c-fca9-4746-bf18-5f6e4109f520 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4575999ff7d64756bef3b338edce2779'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:08:58.371459 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-4ca0323c-fca9-4746-bf18-5f6e4109f520 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4575999ff7d64756bef3b338edce2779'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:08:58.395342 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-4ca0323c-fca9-4746-bf18-5f6e4109f520 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4575999ff7d64756bef3b338edce2779'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:08:58.409464 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-4ca0323c-fca9-4746-bf18-5f6e4109f520 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4575999ff7d64756bef3b338edce2779'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:08:58.410055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-4ca0323c-fca9-4746-bf18-5f6e4109f520 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4575999ff7d64756bef3b338edce2779'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:08:58.415923 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 751/1507] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:08:58 2022] DELETE /identity/v3/users/4575999ff7d64756bef3b338edce2779 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:08:58.420461 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e58ca-4d70-4fce-b1c6-19f6602851ef None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:08:58.420747 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e58ca-4d70-4fce-b1c6-19f6602851ef None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:08:58.421023 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e58ca-4d70-4fce-b1c6-19f6602851ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:08:58.478792 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-022e58ca-4d70-4fce-b1c6-19f6602851ef None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:08:58.480011 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-022e58ca-4d70-4fce-b1c6-19f6602851ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:08:58.556499 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:08:58.570017 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 757/1508] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:08:58.581294 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0323c-fca9-4746-bf18-5f6e4109f520 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:08:58.582570 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-4ca0323c-fca9-4746-bf18-5f6e4109f520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:08:58.682083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0323c-fca9-4746-bf18-5f6e4109f520 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:08:58.736153 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:08:58.757032 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f92b6e-b40f-4775-9400-8ac1d90d03d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:08:58.757647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f92b6e-b40f-4775-9400-8ac1d90d03d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:08:58.757960 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f92b6e-b40f-4775-9400-8ac1d90d03d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:08:58.758283 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f92b6e-b40f-4775-9400-8ac1d90d03d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:08:58.760412 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-b8f92b6e-b40f-4775-9400-8ac1d90d03d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:08:58.873258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f92b6e-b40f-4775-9400-8ac1d90d03d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:08:58.874715 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f92b6e-b40f-4775-9400-8ac1d90d03d1 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:08:58.893760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 752/1509] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:08:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:08:59.266336 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e58ca-4d70-4fce-b1c6-19f6602851ef None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:08:59.267737 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-022e58ca-4d70-4fce-b1c6-19f6602851ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:08:59.279645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e58ca-4d70-4fce-b1c6-19f6602851ef None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:08:59.382935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a070260a-656a-4fa4-92ba-fd37e02e50f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:08:59.384129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070260a-656a-4fa4-92ba-fd37e02e50f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:08:59.384536 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070260a-656a-4fa4-92ba-fd37e02e50f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:08:59.384810 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070260a-656a-4fa4-92ba-fd37e02e50f2 admin admin] PATH_INFO: `/v3/projects/d03f551c5a9d47a29672411a463cb047` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:08:59.395443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a070260a-656a-4fa4-92ba-fd37e02e50f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d03f551c5a9d47a29672411a463cb047)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:08:59.396457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: WARNING py.warnings [None req-a070260a-656a-4fa4-92ba-fd37e02e50f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:08:59.396457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: warnings.warn(msg) Aug 29 20:08:59.396457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: Aug 29 20:08:59.397657 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a070260a-656a-4fa4-92ba-fd37e02e50f2 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:08:59.456684 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a070260a-656a-4fa4-92ba-fd37e02e50f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd03f551c5a9d47a29672411a463cb047', 'request_id': 'req-a070260a-656a-4fa4-92ba-fd37e02e50f2'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:08:59.463525 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a070260a-656a-4fa4-92ba-fd37e02e50f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd03f551c5a9d47a29672411a463cb047', 'request_id': 'req-a070260a-656a-4fa4-92ba-fd37e02e50f2'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:08:59.471752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a070260a-656a-4fa4-92ba-fd37e02e50f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd03f551c5a9d47a29672411a463cb047', 'request_id': 'req-a070260a-656a-4fa4-92ba-fd37e02e50f2'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:08:59.512397 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a070260a-656a-4fa4-92ba-fd37e02e50f2 admin admin] The token cache is being invalidate because project d03f551c5a9d47a29672411a463cb047 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:08:59.512397 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a070260a-656a-4fa4-92ba-fd37e02e50f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:08:59.512397 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a070260a-656a-4fa4-92ba-fd37e02e50f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:08:59.512397 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 758/1510] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:08:59 2022] DELETE /identity/v3/projects/d03f551c5a9d47a29672411a463cb047 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:09:08.891490 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=62 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:09.503650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:10:30.911769 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35c638-4fb8-482a-8412-e5a8c5ff65e8 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:30.912443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35c638-4fb8-482a-8412-e5a8c5ff65e8 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:30.912443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35c638-4fb8-482a-8412-e5a8c5ff65e8 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:30.914205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:10:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:10:30.921316 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7c71f-f3e0-4245-945b-6aeb41718577 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:30.922045 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7c71f-f3e0-4245-945b-6aeb41718577 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:30.922045 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7c71f-f3e0-4245-945b-6aeb41718577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:31.059617 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-b5f7c71f-f3e0-4245-945b-6aeb41718577 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:10:31.060865 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-b5f7c71f-f3e0-4245-945b-6aeb41718577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:10:31.206487 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:10:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:10:31.376414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a35c638-4fb8-482a-8412-e5a8c5ff65e8 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:10:31.379204 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-6a35c638-4fb8-482a-8412-e5a8c5ff65e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:10:31.548495 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a35c638-4fb8-482a-8412-e5a8c5ff65e8 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:10:31.556357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378be089-2845-4a06-a7e9-365b5ca3dd73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:10:31.557276 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378be089-2845-4a06-a7e9-365b5ca3dd73 service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:31.557673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378be089-2845-4a06-a7e9-365b5ca3dd73 service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:31.559411 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378be089-2845-4a06-a7e9-365b5ca3dd73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:31.573759 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378be089-2845-4a06-a7e9-365b5ca3dd73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:10:31.576490 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378be089-2845-4a06-a7e9-365b5ca3dd73 service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:10:31.598372 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 754/1513] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:10:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:10:32.607403 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4535d-06c7-4697-946c-a9febbdc17bf None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:32.608052 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4535d-06c7-4697-946c-a9febbdc17bf None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:32.608052 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4535d-06c7-4697-946c-a9febbdc17bf None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:32.609333 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:10:32.615418 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d388e20-bfe8-4a09-8878-7decc8fa6255 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:32.615993 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d388e20-bfe8-4a09-8878-7decc8fa6255 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:32.616188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d388e20-bfe8-4a09-8878-7decc8fa6255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:32.651171 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-9d388e20-bfe8-4a09-8878-7decc8fa6255 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:10:32.652408 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9d388e20-bfe8-4a09-8878-7decc8fa6255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:10:32.660569 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:10:32.824103 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac4535d-06c7-4697-946c-a9febbdc17bf None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:10:32.840975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac4535d-06c7-4697-946c-a9febbdc17bf None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:10:32.847495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796a519-b427-4683-afb8-86f74b332294 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:10:32.848321 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796a519-b427-4683-afb8-86f74b332294 service nova] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:32.848646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796a519-b427-4683-afb8-86f74b332294 service nova] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:32.848964 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796a519-b427-4683-afb8-86f74b332294 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:32.861454 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d796a519-b427-4683-afb8-86f74b332294 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:10:32.863179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d796a519-b427-4683-afb8-86f74b332294 service nova] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:10:32.875683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 761/1516] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:10:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:10:32.973187 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e8aea-15b1-4ba3-abed-e364e5be118e None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:32.973632 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e8aea-15b1-4ba3-abed-e364e5be118e None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:32.974099 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e8aea-15b1-4ba3-abed-e364e5be118e None None] PATH_INFO: `/v3` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:32.975591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 756/1517] 10.222.0.77 () {54 vars in 925 bytes} [Mon Aug 29 20:10:32 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:10:32.985031 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa95646-8713-4131-bdaf-0499919e84e3 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:32.985493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa95646-8713-4131-bdaf-0499919e84e3 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:32.985941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa95646-8713-4131-bdaf-0499919e84e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:33.069363 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-3fa95646-8713-4131-bdaf-0499919e84e3 None None] MFA Rules not processed for user `1ab2e318b2854c46bc7ed2151a44a2e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:10:33.070775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-3fa95646-8713-4131-bdaf-0499919e84e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:10:33.099684 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:10:33.164316 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 762/1518] 10.222.0.77 () {58 vars in 1015 bytes} [Mon Aug 29 20:10:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:10:42.471568 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e20715-f7e9-415e-a969-abc6824e909f None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:42.472131 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e20715-f7e9-415e-a969-abc6824e909f None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:42.472131 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e20715-f7e9-415e-a969-abc6824e909f None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:42.473400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 757/1519] 10.222.0.73 () {54 vars in 961 bytes} [Mon Aug 29 20:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:10:42.478601 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670696e9-c726-4e88-827b-f12ce1fa6fcc None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:42.479032 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670696e9-c726-4e88-827b-f12ce1fa6fcc None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:42.479133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670696e9-c726-4e88-827b-f12ce1fa6fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:42.499323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-670696e9-c726-4e88-827b-f12ce1fa6fcc None None] MFA Rules not processed for user `1ab2e318b2854c46bc7ed2151a44a2e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:10:42.500525 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-670696e9-c726-4e88-827b-f12ce1fa6fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:10:42.509993 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 763/1520] 10.222.0.73 () {58 vars in 1058 bytes} [Mon Aug 29 20:10:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:10:42.517828 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e20715-f7e9-415e-a969-abc6824e909f None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:10:42.531567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e20715-f7e9-415e-a969-abc6824e909f None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:10:42.536571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a48a8-9417-4073-b301-6725e67fa263 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab2e318b2854c46bc7ed2151a44a2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:10:42.537160 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a48a8-9417-4073-b301-6725e67fa263 service placement] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:42.537442 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a48a8-9417-4073-b301-6725e67fa263 service placement] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:42.537726 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a48a8-9417-4073-b301-6725e67fa263 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:42.546174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a48a8-9417-4073-b301-6725e67fa263 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:10:42.547776 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a48a8-9417-4073-b301-6725e67fa263 service placement] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:10:42.557901 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 758/1521] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:10:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:10:44.118184 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84b9eb-19a8-47f2-9a97-e88c3fd87497 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:44.118690 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84b9eb-19a8-47f2-9a97-e88c3fd87497 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:44.118813 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84b9eb-19a8-47f2-9a97-e88c3fd87497 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:44.120109 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:10:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:10:44.124857 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa00f54-78e2-45af-bd95-b22eea7626bf None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:44.125266 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa00f54-78e2-45af-bd95-b22eea7626bf None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:44.125408 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa00f54-78e2-45af-bd95-b22eea7626bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:44.145317 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-6aa00f54-78e2-45af-bd95-b22eea7626bf None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:10:44.146537 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-6aa00f54-78e2-45af-bd95-b22eea7626bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:10:44.154613 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:10:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:10:44.307889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f84b9eb-19a8-47f2-9a97-e88c3fd87497 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:10:44.322444 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f84b9eb-19a8-47f2-9a97-e88c3fd87497 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:10:44.327902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb21c1-40b9-47ed-9a00-eaeab22a0e1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:10:44.328495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb21c1-40b9-47ed-9a00-eaeab22a0e1d service nova] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:44.328776 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb21c1-40b9-47ed-9a00-eaeab22a0e1d service nova] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:44.329048 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb21c1-40b9-47ed-9a00-eaeab22a0e1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:44.337623 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb21c1-40b9-47ed-9a00-eaeab22a0e1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:10:44.339177 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb21c1-40b9-47ed-9a00-eaeab22a0e1d service nova] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:10:44.348660 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 765/1524] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:10:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:10:45.168136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9146c9b-7890-4f36-a7f7-3cadd551f42e None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:45.168696 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9146c9b-7890-4f36-a7f7-3cadd551f42e None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:45.168696 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9146c9b-7890-4f36-a7f7-3cadd551f42e None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:45.169981 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:10:45.174744 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc6029-c36b-42e5-90c7-31a53389c5d5 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:45.175516 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc6029-c36b-42e5-90c7-31a53389c5d5 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:45.175974 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc6029-c36b-42e5-90c7-31a53389c5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:45.196261 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-47fc6029-c36b-42e5-90c7-31a53389c5d5 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:10:45.197463 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-47fc6029-c36b-42e5-90c7-31a53389c5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:10:45.205412 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:10:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:10:45.357084 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9146c9b-7890-4f36-a7f7-3cadd551f42e None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:10:45.372723 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9146c9b-7890-4f36-a7f7-3cadd551f42e None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:10:45.378286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48405f4e-64b4-4666-bf64-9c8f1bc334d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:10:45.378882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48405f4e-64b4-4666-bf64-9c8f1bc334d4 service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:45.379163 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48405f4e-64b4-4666-bf64-9c8f1bc334d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:45.379434 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48405f4e-64b4-4666-bf64-9c8f1bc334d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:45.387890 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48405f4e-64b4-4666-bf64-9c8f1bc334d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:10:45.389461 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48405f4e-64b4-4666-bf64-9c8f1bc334d4 service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:10:45.399657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 761/1527] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:10:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:10:49.644859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d652de-592e-4beb-b116-bc73ad0642c9 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:49.645440 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d652de-592e-4beb-b116-bc73ad0642c9 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:49.645440 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d652de-592e-4beb-b116-bc73ad0642c9 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:49.646784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:10:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:10:49.652558 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e435aa1-3de0-476f-b44f-e37699ee3080 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:49.653040 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e435aa1-3de0-476f-b44f-e37699ee3080 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:49.653148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e435aa1-3de0-476f-b44f-e37699ee3080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:49.673469 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-1e435aa1-3de0-476f-b44f-e37699ee3080 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:10:49.674708 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-1e435aa1-3de0-476f-b44f-e37699ee3080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:10:49.682601 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:10:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:10:49.971057 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d652de-592e-4beb-b116-bc73ad0642c9 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:10:49.972808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-e1d652de-592e-4beb-b116-bc73ad0642c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:10:50.052438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d652de-592e-4beb-b116-bc73ad0642c9 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:10:50.056650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43097ae6-363d-4eff-a13d-ec807da4afe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:10:50.057251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43097ae6-363d-4eff-a13d-ec807da4afe3 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:50.057531 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43097ae6-363d-4eff-a13d-ec807da4afe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:50.057831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43097ae6-363d-4eff-a13d-ec807da4afe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:50.066803 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43097ae6-363d-4eff-a13d-ec807da4afe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:10:50.068230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43097ae6-363d-4eff-a13d-ec807da4afe3 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:10:50.078344 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 768/1530] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:10:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:10:51.990264 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8471f46-095d-4213-be98-cac369b5a8bd None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:51.990844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8471f46-095d-4213-be98-cac369b5a8bd None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:51.990844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8471f46-095d-4213-be98-cac369b5a8bd None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:51.992972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:10:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:10:51.998392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8676f-47fd-45c6-8c9e-dcd1a07aaefa None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:51.998807 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8676f-47fd-45c6-8c9e-dcd1a07aaefa None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:51.998941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8676f-47fd-45c6-8c9e-dcd1a07aaefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:52.018845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-aec8676f-47fd-45c6-8c9e-dcd1a07aaefa None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:10:52.020056 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-aec8676f-47fd-45c6-8c9e-dcd1a07aaefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:10:52.027942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:10:52.322612 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8471f46-095d-4213-be98-cac369b5a8bd None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:10:52.333389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8471f46-095d-4213-be98-cac369b5a8bd None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:10:52.337577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6404366-0b2c-4bf7-9d00-b2439c60c1fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:10:52.338207 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6404366-0b2c-4bf7-9d00-b2439c60c1fe service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:52.338490 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6404366-0b2c-4bf7-9d00-b2439c60c1fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:52.338761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6404366-0b2c-4bf7-9d00-b2439c60c1fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:52.347793 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6404366-0b2c-4bf7-9d00-b2439c60c1fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:10:52.349216 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6404366-0b2c-4bf7-9d00-b2439c60c1fe service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:10:52.358754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 764/1533] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:10:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:10:56.077233 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61b4d8-df45-4d32-92c3-2e26843efd93 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:56.077762 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61b4d8-df45-4d32-92c3-2e26843efd93 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:56.077762 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61b4d8-df45-4d32-92c3-2e26843efd93 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:56.079065 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:10:56.084420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24f03e-00ab-46a4-b68b-b59f1780cbcc None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:56.084860 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24f03e-00ab-46a4-b68b-b59f1780cbcc None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:56.084956 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24f03e-00ab-46a4-b68b-b59f1780cbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:56.106859 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-de24f03e-00ab-46a4-b68b-b59f1780cbcc None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:10:56.108083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-de24f03e-00ab-46a4-b68b-b59f1780cbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:10:56.116014 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:10:56.668318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61b4d8-df45-4d32-92c3-2e26843efd93 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:10:56.670236 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-6d61b4d8-df45-4d32-92c3-2e26843efd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:10:56.737607 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:10:56.751611 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61b4d8-df45-4d32-92c3-2e26843efd93 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:10:56.755187 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c9d5e-c206-4cfe-b3ca-113bbd3d404a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:10:56.755794 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c9d5e-c206-4cfe-b3ca-113bbd3d404a service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:56.756072 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c9d5e-c206-4cfe-b3ca-113bbd3d404a service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:56.756345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c9d5e-c206-4cfe-b3ca-113bbd3d404a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:56.766330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c9d5e-c206-4cfe-b3ca-113bbd3d404a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:10:56.767751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c9d5e-c206-4cfe-b3ca-113bbd3d404a service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:10:56.777149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 771/1536] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:10:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:10:58.715194 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ab954-8900-4af6-901e-2c82822748e3 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:58.715755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ab954-8900-4af6-901e-2c82822748e3 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:58.715892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ab954-8900-4af6-901e-2c82822748e3 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:58.717255 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:10:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:10:58.722604 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca3d30-9bdf-4c56-96ef-3edc73f8f008 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:58.723027 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca3d30-9bdf-4c56-96ef-3edc73f8f008 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:58.723134 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca3d30-9bdf-4c56-96ef-3edc73f8f008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:58.742828 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-c9ca3d30-9bdf-4c56-96ef-3edc73f8f008 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:10:58.744110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c9ca3d30-9bdf-4c56-96ef-3edc73f8f008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:10:58.752547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:10:59.299654 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ab954-8900-4af6-901e-2c82822748e3 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:10:59.301334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-5f4ab954-8900-4af6-901e-2c82822748e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:10:59.309487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=84 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:10:59.316013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ab954-8900-4af6-901e-2c82822748e3 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:10:59.319548 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b74610-3327-4f00-b56a-7d1ae8d44bb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:10:59.320132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b74610-3327-4f00-b56a-7d1ae8d44bb8 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:10:59.320410 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b74610-3327-4f00-b56a-7d1ae8d44bb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:10:59.320732 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b74610-3327-4f00-b56a-7d1ae8d44bb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:10:59.329994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b74610-3327-4f00-b56a-7d1ae8d44bb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:10:59.331415 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b74610-3327-4f00-b56a-7d1ae8d44bb8 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:10:59.340851 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 767/1539] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:10:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:11:08.737969 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=11 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:11:09.336558 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=5 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:25.308955 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40255e-ef70-45a3-9862-0bbc03d39e6d None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:25.309576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40255e-ef70-45a3-9862-0bbc03d39e6d None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:25.309576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40255e-ef70-45a3-9862-0bbc03d39e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:25.346540 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-9f40255e-ef70-45a3-9862-0bbc03d39e6d None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:25.347968 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-9f40255e-ef70-45a3-9862-0bbc03d39e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:25.361777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 773/1540] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:25.370924 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579bf3b-2492-4215-80e5-46f321c91a94 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:25.371993 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579bf3b-2492-4215-80e5-46f321c91a94 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:25.373195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579bf3b-2492-4215-80e5-46f321c91a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:25.407456 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-f579bf3b-2492-4215-80e5-46f321c91a94 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:25.409521 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-f579bf3b-2492-4215-80e5-46f321c91a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:25.428226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 768/1541] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:25.434280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40255e-ef70-45a3-9862-0bbc03d39e6d None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:25.459095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40255e-ef70-45a3-9862-0bbc03d39e6d None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:25.471329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2baf1-a06c-4648-8ee5-db3faf3e408a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:25.471963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2baf1-a06c-4648-8ee5-db3faf3e408a admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:25.472245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2baf1-a06c-4648-8ee5-db3faf3e408a admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:25.472519 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2baf1-a06c-4648-8ee5-db3faf3e408a admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:25.473176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb2baf1-a06c-4648-8ee5-db3faf3e408a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:12:25.473476 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb2baf1-a06c-4648-8ee5-db3faf3e408a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:25.475123 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb2baf1-a06c-4648-8ee5-db3faf3e408a admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:25.493267 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 774/1542] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:12:25.502817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579bf3b-2492-4215-80e5-46f321c91a94 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:25.517735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579bf3b-2492-4215-80e5-46f321c91a94 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:25.530072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab3395-704c-4806-a742-aebc6b1eb518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:25.531046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab3395-704c-4806-a742-aebc6b1eb518 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:25.531539 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab3395-704c-4806-a742-aebc6b1eb518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:25.532180 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab3395-704c-4806-a742-aebc6b1eb518 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:25.533034 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab3395-704c-4806-a742-aebc6b1eb518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:25.535169 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab3395-704c-4806-a742-aebc6b1eb518 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:25.594841 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 769/1543] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:25 2022] POST /identity/v3/projects => generated 371 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:25.599555 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2baf1-a06c-4648-8ee5-db3faf3e408a admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:25.614696 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2baf1-a06c-4648-8ee5-db3faf3e408a admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:25.625133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401e89a-d8c0-4448-b68c-9eae2935550b None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:25.625781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401e89a-d8c0-4448-b68c-9eae2935550b None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:25.626274 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401e89a-d8c0-4448-b68c-9eae2935550b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:25.659116 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-9401e89a-d8c0-4448-b68c-9eae2935550b None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:25.661423 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9401e89a-d8c0-4448-b68c-9eae2935550b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:25.709386 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb1ee5-f1e2-4739-b5a1-33a5cc025df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:25.710056 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb1ee5-f1e2-4739-b5a1-33a5cc025df3 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:25.710339 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb1ee5-f1e2-4739-b5a1-33a5cc025df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:25.710615 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb1ee5-f1e2-4739-b5a1-33a5cc025df3 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:25.711261 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb1ee5-f1e2-4739-b5a1-33a5cc025df3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:25.718249 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb1ee5-f1e2-4739-b5a1-33a5cc025df3 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:25.750638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 775/1544] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:25 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:25.752962 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07c276-781d-47ee-9d55-1482cd666eba None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:25.753480 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07c276-781d-47ee-9d55-1482cd666eba None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:25.754146 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07c276-781d-47ee-9d55-1482cd666eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:25.775573 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 770/1545] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:25.776916 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b4589-e55b-4106-a1c2-91a676c04edf None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:25.777210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b4589-e55b-4106-a1c2-91a676c04edf None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:25.777488 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b4589-e55b-4106-a1c2-91a676c04edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:25.780832 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-fc07c276-781d-47ee-9d55-1482cd666eba None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:25.782330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-fc07c276-781d-47ee-9d55-1482cd666eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:25.793595 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 776/1546] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:25.796307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088af36-dd0f-4d1f-83c8-af077ab00016 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:25.796762 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088af36-dd0f-4d1f-83c8-af077ab00016 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:25.797193 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088af36-dd0f-4d1f-83c8-af077ab00016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:25.805936 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-e45b4589-e55b-4106-a1c2-91a676c04edf None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:25.807144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-e45b4589-e55b-4106-a1c2-91a676c04edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:25.815470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 771/1547] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:25.816762 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80160a7-94e9-4ec6-bb34-754cff8beafa None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:25.817048 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-5088af36-dd0f-4d1f-83c8-af077ab00016 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:25.817184 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80160a7-94e9-4ec6-bb34-754cff8beafa None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:25.817366 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80160a7-94e9-4ec6-bb34-754cff8beafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:25.818246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-5088af36-dd0f-4d1f-83c8-af077ab00016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:25.830140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 777/1548] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:25.830970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088af36-dd0f-4d1f-83c8-af077ab00016 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:25.842084 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088af36-dd0f-4d1f-83c8-af077ab00016 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:25.845257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-d80160a7-94e9-4ec6-bb34-754cff8beafa None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:25.846456 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d80160a7-94e9-4ec6-bb34-754cff8beafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:25.846740 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48010a1-7dd4-4f09-8bdc-c6be1ebe9519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:25.847348 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48010a1-7dd4-4f09-8bdc-c6be1ebe9519 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:25.847756 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48010a1-7dd4-4f09-8bdc-c6be1ebe9519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:25.848038 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48010a1-7dd4-4f09-8bdc-c6be1ebe9519 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:25.848715 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48010a1-7dd4-4f09-8bdc-c6be1ebe9519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:25.850411 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48010a1-7dd4-4f09-8bdc-c6be1ebe9519 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:25.854434 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 772/1549] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:25.855619 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e19f4-93cb-44a2-95eb-cbe23ea17ae7 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:25.855901 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e19f4-93cb-44a2-95eb-cbe23ea17ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:25.856172 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e19f4-93cb-44a2-95eb-cbe23ea17ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:25.862433 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 778/1550] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:25.863720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845c8fe-ce50-4fbf-9b7f-35a0dfc3fb8f None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:25.864269 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845c8fe-ce50-4fbf-9b7f-35a0dfc3fb8f None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:25.864624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845c8fe-ce50-4fbf-9b7f-35a0dfc3fb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:25.882164 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-7f8e19f4-93cb-44a2-95eb-cbe23ea17ae7 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:25.883352 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-7f8e19f4-93cb-44a2-95eb-cbe23ea17ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:25.883808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-f845c8fe-ce50-4fbf-9b7f-35a0dfc3fb8f None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:25.884991 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f845c8fe-ce50-4fbf-9b7f-35a0dfc3fb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:25.891742 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 773/1551] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:25.892912 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd34768-eeb9-4167-92db-2316e6980c37 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:25.893188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd34768-eeb9-4167-92db-2316e6980c37 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:25.893451 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd34768-eeb9-4167-92db-2316e6980c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:25.894377 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 779/1552] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:25.895547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57ea08-e704-4ff3-aee2-a25bcc6f3b41 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:25.895832 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57ea08-e704-4ff3-aee2-a25bcc6f3b41 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:25.896104 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57ea08-e704-4ff3-aee2-a25bcc6f3b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:25.916301 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-abd34768-eeb9-4167-92db-2316e6980c37 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:25.917486 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-abd34768-eeb9-4167-92db-2316e6980c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:25.921073 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-2d57ea08-e704-4ff3-aee2-a25bcc6f3b41 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:25.922271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-2d57ea08-e704-4ff3-aee2-a25bcc6f3b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:25.926176 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 774/1553] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:25.927344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2fc26b-7fa5-43ba-a8e8-1ae8f35f3976 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:25.927621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2fc26b-7fa5-43ba-a8e8-1ae8f35f3976 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:25.927887 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2fc26b-7fa5-43ba-a8e8-1ae8f35f3976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:25.930450 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 780/1554] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:25.931650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928373a-6462-43da-a497-89c58d92e817 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:25.931930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928373a-6462-43da-a497-89c58d92e817 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:25.932199 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928373a-6462-43da-a497-89c58d92e817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:25.951413 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-db2fc26b-7fa5-43ba-a8e8-1ae8f35f3976 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:25.951650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-c928373a-6462-43da-a497-89c58d92e817 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:25.952632 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c928373a-6462-43da-a497-89c58d92e817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:25.952791 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-db2fc26b-7fa5-43ba-a8e8-1ae8f35f3976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:25.960417 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 781/1555] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:25.964079 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 775/1556] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:25.964201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928373a-6462-43da-a497-89c58d92e817 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:25.964293 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c2cf8-ad26-44a1-a935-c716354e5a68 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:25.964293 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c2cf8-ad26-44a1-a935-c716354e5a68 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:25.964293 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c2cf8-ad26-44a1-a935-c716354e5a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:25.974616 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928373a-6462-43da-a497-89c58d92e817 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:25.979208 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f7051-89ed-49e1-bfeb-a969de44ddb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:25.979801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f7051-89ed-49e1-bfeb-a969de44ddb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:25.980077 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f7051-89ed-49e1-bfeb-a969de44ddb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:25.980346 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f7051-89ed-49e1-bfeb-a969de44ddb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:25.980984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f7051-89ed-49e1-bfeb-a969de44ddb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:25.983044 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-548c2cf8-ad26-44a1-a935-c716354e5a68 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:25.983208 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f7051-89ed-49e1-bfeb-a969de44ddb8 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:25.984249 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-548c2cf8-ad26-44a1-a935-c716354e5a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:25.992410 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 776/1557] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:25.996203 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 782/1558] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:25.996376 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5affe31-1d67-487d-ae16-f6bede68d939 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:25.996376 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5affe31-1d67-487d-ae16-f6bede68d939 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:25.996578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f7051-89ed-49e1-bfeb-a969de44ddb8 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:25.996689 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5affe31-1d67-487d-ae16-f6bede68d939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.006845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f7051-89ed-49e1-bfeb-a969de44ddb8 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:26.012512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761efee9-1887-4304-96ae-83434a3fcd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:26.013444 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761efee9-1887-4304-96ae-83434a3fcd3f admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.013963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761efee9-1887-4304-96ae-83434a3fcd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.014144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-e5affe31-1d67-487d-ae16-f6bede68d939 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:26.014457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761efee9-1887-4304-96ae-83434a3fcd3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.015345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761efee9-1887-4304-96ae-83434a3fcd3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:12:26.015495 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-e5affe31-1d67-487d-ae16-f6bede68d939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:26.015753 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761efee9-1887-4304-96ae-83434a3fcd3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:26.018174 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761efee9-1887-4304-96ae-83434a3fcd3f admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:26.023924 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 777/1559] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:26.025032 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e5173-7d90-4f1e-9a5d-6189446519c0 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.025393 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e5173-7d90-4f1e-9a5d-6189446519c0 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.025672 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e5173-7d90-4f1e-9a5d-6189446519c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.038561 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 783/1560] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:12:26.038561 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761efee9-1887-4304-96ae-83434a3fcd3f admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:26.049878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-577e5173-7d90-4f1e-9a5d-6189446519c0 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:26.051088 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-577e5173-7d90-4f1e-9a5d-6189446519c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:26.052439 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761efee9-1887-4304-96ae-83434a3fcd3f admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:26.059088 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 778/1561] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:26.060029 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e5173-7d90-4f1e-9a5d-6189446519c0 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:26.070278 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e5173-7d90-4f1e-9a5d-6189446519c0 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:26.071512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc9b55-7a07-492a-8b71-3a1a408e556d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:26.072126 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc9b55-7a07-492a-8b71-3a1a408e556d admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.072402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc9b55-7a07-492a-8b71-3a1a408e556d admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.072674 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc9b55-7a07-492a-8b71-3a1a408e556d admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.073342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc9b55-7a07-492a-8b71-3a1a408e556d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:12:26.073692 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc9b55-7a07-492a-8b71-3a1a408e556d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:26.075036 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7562c35f-d65a-4787-9b5e-547e777ac219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:26.075323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc9b55-7a07-492a-8b71-3a1a408e556d admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:26.076145 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562c35f-d65a-4787-9b5e-547e777ac219 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.076427 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562c35f-d65a-4787-9b5e-547e777ac219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.076701 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562c35f-d65a-4787-9b5e-547e777ac219 admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.077339 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7562c35f-d65a-4787-9b5e-547e777ac219 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:12:26.077649 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7562c35f-d65a-4787-9b5e-547e777ac219 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:26.078685 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: WARNING py.warnings [None req-7562c35f-d65a-4787-9b5e-547e777ac219 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:12:26.078685 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: warnings.warn(msg) Aug 29 20:12:26.078685 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: Aug 29 20:12:26.079559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7562c35f-d65a-4787-9b5e-547e777ac219 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:26.088619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 784/1562] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:12:26.089452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc9b55-7a07-492a-8b71-3a1a408e556d admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:26.102272 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc9b55-7a07-492a-8b71-3a1a408e556d admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:26.103647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 779/1563] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:12:26.104497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7562c35f-d65a-4787-9b5e-547e777ac219 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:26.106878 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41ce9f-d407-42d1-834a-5c9212a1d8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:26.107874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41ce9f-d407-42d1-834a-5c9212a1d8ca admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.108204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41ce9f-d407-42d1-834a-5c9212a1d8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.108503 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41ce9f-d407-42d1-834a-5c9212a1d8ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.109143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41ce9f-d407-42d1-834a-5c9212a1d8ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:12:26.109464 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41ce9f-d407-42d1-834a-5c9212a1d8ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:26.111491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41ce9f-d407-42d1-834a-5c9212a1d8ca admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:26.115079 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7562c35f-d65a-4787-9b5e-547e777ac219 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:26.119794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40932007-e5a3-444a-b1d9-cbc44202ad20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:26.120509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40932007-e5a3-444a-b1d9-cbc44202ad20 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.120837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40932007-e5a3-444a-b1d9-cbc44202ad20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.121175 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40932007-e5a3-444a-b1d9-cbc44202ad20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.121938 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40932007-e5a3-444a-b1d9-cbc44202ad20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:12:26.122325 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40932007-e5a3-444a-b1d9-cbc44202ad20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:26.123772 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 785/1564] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:12:26.124295 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40932007-e5a3-444a-b1d9-cbc44202ad20 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:26.124554 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41ce9f-d407-42d1-834a-5c9212a1d8ca admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:26.134355 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41ce9f-d407-42d1-834a-5c9212a1d8ca admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:26.136944 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 780/1565] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:12:26.137854 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40932007-e5a3-444a-b1d9-cbc44202ad20 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:26.146733 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f552ae8-1d45-4335-bf58-235e2bbed1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:26.147404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f552ae8-1d45-4335-bf58-235e2bbed1ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.147683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f552ae8-1d45-4335-bf58-235e2bbed1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.147957 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f552ae8-1d45-4335-bf58-235e2bbed1ce admin admin] PATH_INFO: `/v3/projects/e02f019a5ffa4653abf9f7633e4251fb/users/103fab04912b42a989f6fa75aa5e4e33/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.148251 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40932007-e5a3-444a-b1d9-cbc44202ad20 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:26.153198 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e348199-679e-42c3-9454-b1e9eacb52a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:26.154849 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e348199-679e-42c3-9454-b1e9eacb52a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.155137 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e348199-679e-42c3-9454-b1e9eacb52a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.155415 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e348199-679e-42c3-9454-b1e9eacb52a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.156061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e348199-679e-42c3-9454-b1e9eacb52a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:12:26.156370 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e348199-679e-42c3-9454-b1e9eacb52a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:26.157994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e348199-679e-42c3-9454-b1e9eacb52a4 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:26.164426 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f552ae8-1d45-4335-bf58-235e2bbed1ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=e02f019a5ffa4653abf9f7633e4251fb, user_id=103fab04912b42a989f6fa75aa5e4e33, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:26.165473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: WARNING py.warnings [None req-9f552ae8-1d45-4335-bf58-235e2bbed1ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:12:26.165473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: warnings.warn(msg) Aug 29 20:12:26.165473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: Aug 29 20:12:26.167462 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f552ae8-1d45-4335-bf58-235e2bbed1ce admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:26.169729 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 781/1566] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:12:26.170518 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e348199-679e-42c3-9454-b1e9eacb52a4 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:26.180287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e348199-679e-42c3-9454-b1e9eacb52a4 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:26.185433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3491d63-08cc-48df-bdb3-5c564828d117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:26.186085 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3491d63-08cc-48df-bdb3-5c564828d117 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.186368 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3491d63-08cc-48df-bdb3-5c564828d117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.186644 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3491d63-08cc-48df-bdb3-5c564828d117 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.187263 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3491d63-08cc-48df-bdb3-5c564828d117 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:26.188670 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 786/1567] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:26 2022] PUT /identity/v3/projects/e02f019a5ffa4653abf9f7633e4251fb/users/103fab04912b42a989f6fa75aa5e4e33/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:26.189148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3491d63-08cc-48df-bdb3-5c564828d117 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:26.189559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f552ae8-1d45-4335-bf58-235e2bbed1ce admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:26.203611 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:26.227267 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 782/1568] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:26 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:26.227267 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3491d63-08cc-48df-bdb3-5c564828d117 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:26.236866 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:26.262204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f552ae8-1d45-4335-bf58-235e2bbed1ce admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:26.298616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3491d63-08cc-48df-bdb3-5c564828d117 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:26.303724 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb11719-1d4d-4b63-9aa9-0d1428ce8271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:26.304702 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb11719-1d4d-4b63-9aa9-0d1428ce8271 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.305179 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb11719-1d4d-4b63-9aa9-0d1428ce8271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.305622 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb11719-1d4d-4b63-9aa9-0d1428ce8271 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.306450 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb11719-1d4d-4b63-9aa9-0d1428ce8271 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:26.308541 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb11719-1d4d-4b63-9aa9-0d1428ce8271 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:26.327768 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96fecb-a197-4189-860c-c998ecb90cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:26.328438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96fecb-a197-4189-860c-c998ecb90cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.328655 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96fecb-a197-4189-860c-c998ecb90cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.329056 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96fecb-a197-4189-860c-c998ecb90cc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.329831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be96fecb-a197-4189-860c-c998ecb90cc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:12:26.330210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be96fecb-a197-4189-860c-c998ecb90cc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:26.332129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be96fecb-a197-4189-860c-c998ecb90cc2 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:26.338603 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 783/1569] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:26 2022] POST /identity/v3/projects => generated 403 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:26.343687 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb11719-1d4d-4b63-9aa9-0d1428ce8271 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:26.345930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 787/1570] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:12:26.346994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96fecb-a197-4189-860c-c998ecb90cc2 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:26.364699 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96fecb-a197-4189-860c-c998ecb90cc2 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:26.423307 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb11719-1d4d-4b63-9aa9-0d1428ce8271 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:26.428638 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a1f82-8d10-4cfb-9b5f-5e57eb58a18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:26.429327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a1f82-8d10-4cfb-9b5f-5e57eb58a18b admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.429871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a1f82-8d10-4cfb-9b5f-5e57eb58a18b admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.430152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a1f82-8d10-4cfb-9b5f-5e57eb58a18b admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.430792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a1f82-8d10-4cfb-9b5f-5e57eb58a18b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:26.431203 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c968d44-0f16-4d4a-9209-6f9098a57a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:26.431809 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c968d44-0f16-4d4a-9209-6f9098a57a73 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.432088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c968d44-0f16-4d4a-9209-6f9098a57a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.432363 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c968d44-0f16-4d4a-9209-6f9098a57a73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.432708 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a1f82-8d10-4cfb-9b5f-5e57eb58a18b admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:26.432979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c968d44-0f16-4d4a-9209-6f9098a57a73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:26.434103 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: WARNING py.warnings [None req-7c968d44-0f16-4d4a-9209-6f9098a57a73 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:12:26.434103 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: warnings.warn(msg) Aug 29 20:12:26.434103 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: Aug 29 20:12:26.436400 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c968d44-0f16-4d4a-9209-6f9098a57a73 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:26.465805 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 784/1571] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:26 2022] POST /identity/v3/projects => generated 393 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:26.469802 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a1f82-8d10-4cfb-9b5f-5e57eb58a18b admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:26.469937 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 788/1572] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:26 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:26.469937 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c968d44-0f16-4d4a-9209-6f9098a57a73 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:26.512227 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:26.542978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a1f82-8d10-4cfb-9b5f-5e57eb58a18b admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:26.543944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c968d44-0f16-4d4a-9209-6f9098a57a73 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:26.548336 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cadada-79e6-462b-bd8c-4a4cd779d739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:26.549111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e59beef-2792-4ae3-9887-1c778fce7d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:26.549263 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cadada-79e6-462b-bd8c-4a4cd779d739 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.549557 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cadada-79e6-462b-bd8c-4a4cd779d739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.549919 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cadada-79e6-462b-bd8c-4a4cd779d739 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.550024 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59beef-2792-4ae3-9887-1c778fce7d52 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.550325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59beef-2792-4ae3-9887-1c778fce7d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.550750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59beef-2792-4ae3-9887-1c778fce7d52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.550862 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cadada-79e6-462b-bd8c-4a4cd779d739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:26.551595 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e59beef-2792-4ae3-9887-1c778fce7d52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:26.553103 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cadada-79e6-462b-bd8c-4a4cd779d739 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:26.553851 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e59beef-2792-4ae3-9887-1c778fce7d52 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:26.585337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 789/1573] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:26 2022] POST /identity/v3/projects => generated 369 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:26.585850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 785/1574] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:26 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:26.589550 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cadada-79e6-462b-bd8c-4a4cd779d739 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:26.590404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68def69-0dac-46d4-b6ec-624b230f2d58 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.590632 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68def69-0dac-46d4-b6ec-624b230f2d58 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.590903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68def69-0dac-46d4-b6ec-624b230f2d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.596355 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:26.623611 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-c68def69-0dac-46d4-b6ec-624b230f2d58 None None] MFA Rules not processed for user `103fab04912b42a989f6fa75aa5e4e33`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:26.624748 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c68def69-0dac-46d4-b6ec-624b230f2d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:26.661187 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cadada-79e6-462b-bd8c-4a4cd779d739 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:26.692695 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 790/1575] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:26.693570 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68def69-0dac-46d4-b6ec-624b230f2d58 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:26.721253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de088c99-68d6-4301-a45f-da345a044c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:26.721978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de088c99-68d6-4301-a45f-da345a044c1a admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.722259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de088c99-68d6-4301-a45f-da345a044c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.722532 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de088c99-68d6-4301-a45f-da345a044c1a admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.723160 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de088c99-68d6-4301-a45f-da345a044c1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:26.724173 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: WARNING py.warnings [None req-de088c99-68d6-4301-a45f-da345a044c1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:12:26.724173 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: warnings.warn(msg) Aug 29 20:12:26.724173 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: Aug 29 20:12:26.725368 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de088c99-68d6-4301-a45f-da345a044c1a admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:26.749426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 786/1576] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:26 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:26.750240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de088c99-68d6-4301-a45f-da345a044c1a admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:26.760352 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de088c99-68d6-4301-a45f-da345a044c1a admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:26.761259 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68def69-0dac-46d4-b6ec-624b230f2d58 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:26.766008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d26a14-63d2-45e9-bf2c-8fc6897c5217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:26.766706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d26a14-63d2-45e9-bf2c-8fc6897c5217 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.766834 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1f4e2-5bb7-4fa7-89c6-c9507fc53bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:26.767006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d26a14-63d2-45e9-bf2c-8fc6897c5217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.767274 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d26a14-63d2-45e9-bf2c-8fc6897c5217 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.767378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1f4e2-5bb7-4fa7-89c6-c9507fc53bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.767575 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1f4e2-5bb7-4fa7-89c6-c9507fc53bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.767849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1f4e2-5bb7-4fa7-89c6-c9507fc53bd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.767948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d26a14-63d2-45e9-bf2c-8fc6897c5217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:26.768473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1f4e2-5bb7-4fa7-89c6-c9507fc53bd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:26.770197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d26a14-63d2-45e9-bf2c-8fc6897c5217 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:26.770801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1f4e2-5bb7-4fa7-89c6-c9507fc53bd1 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:26.792878 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 791/1577] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:26 2022] POST /identity/v3/users => generated 302 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:26.794059 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1f4e2-5bb7-4fa7-89c6-c9507fc53bd1 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:26.798604 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 787/1578] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:26 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:26.799775 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d26a14-63d2-45e9-bf2c-8fc6897c5217 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:26.806918 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1f4e2-5bb7-4fa7-89c6-c9507fc53bd1 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:26.855770 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:26.871171 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d26a14-63d2-45e9-bf2c-8fc6897c5217 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:26.874481 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c38a6ce-8403-4e1e-81cd-c50319ddd261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:26.875043 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38a6ce-8403-4e1e-81cd-c50319ddd261 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.875321 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38a6ce-8403-4e1e-81cd-c50319ddd261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.875591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38a6ce-8403-4e1e-81cd-c50319ddd261 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.876226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c38a6ce-8403-4e1e-81cd-c50319ddd261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:26.877586 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e88eec-f062-453f-8485-bbd72c8bcd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:26.878190 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c38a6ce-8403-4e1e-81cd-c50319ddd261 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:26.878395 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e88eec-f062-453f-8485-bbd72c8bcd61 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.878557 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e88eec-f062-453f-8485-bbd72c8bcd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.878920 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e88eec-f062-453f-8485-bbd72c8bcd61 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.879542 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e88eec-f062-453f-8485-bbd72c8bcd61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:26.881770 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e88eec-f062-453f-8485-bbd72c8bcd61 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:26.901011 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 792/1579] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:26 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:26.904989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c38a6ce-8403-4e1e-81cd-c50319ddd261 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:26.915938 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 788/1580] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:26 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:26.915938 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e88eec-f062-453f-8485-bbd72c8bcd61 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:26.916287 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c38a6ce-8403-4e1e-81cd-c50319ddd261 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:26.919927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2155fe6-5788-453a-9753-4012ad4f6d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:26.920537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2155fe6-5788-453a-9753-4012ad4f6d6a admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.920804 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2155fe6-5788-453a-9753-4012ad4f6d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.921069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2155fe6-5788-453a-9753-4012ad4f6d6a admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.921685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2155fe6-5788-453a-9753-4012ad4f6d6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:26.923622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2155fe6-5788-453a-9753-4012ad4f6d6a admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:26.925270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e88eec-f062-453f-8485-bbd72c8bcd61 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:26.929914 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cacebc-573a-40c3-89a3-94e087e952cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:26.930493 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cacebc-573a-40c3-89a3-94e087e952cb admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.930775 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cacebc-573a-40c3-89a3-94e087e952cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.931050 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cacebc-573a-40c3-89a3-94e087e952cb admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.931671 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cacebc-573a-40c3-89a3-94e087e952cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:26.933835 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cacebc-573a-40c3-89a3-94e087e952cb admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:26.944534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:26.948484 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 793/1581] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:26 2022] POST /identity/v3/users => generated 285 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:26.949430 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2155fe6-5788-453a-9753-4012ad4f6d6a admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:26.960928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 789/1582] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:26 2022] POST /identity/v3/users => generated 289 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:26.961168 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2155fe6-5788-453a-9753-4012ad4f6d6a admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:26.961634 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cacebc-573a-40c3-89a3-94e087e952cb admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:26.966069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47e433-b9a1-4547-95c2-f98cafa25daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:26.966892 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47e433-b9a1-4547-95c2-f98cafa25daf admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.967158 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47e433-b9a1-4547-95c2-f98cafa25daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.967437 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47e433-b9a1-4547-95c2-f98cafa25daf admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.968051 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47e433-b9a1-4547-95c2-f98cafa25daf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:26.970000 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47e433-b9a1-4547-95c2-f98cafa25daf admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:26.971294 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cacebc-573a-40c3-89a3-94e087e952cb admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:26.975954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da0ae6-c831-40e4-9ab8-f2f8dd93af21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:26.976552 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da0ae6-c831-40e4-9ab8-f2f8dd93af21 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:26.976828 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da0ae6-c831-40e4-9ab8-f2f8dd93af21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:26.977104 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da0ae6-c831-40e4-9ab8-f2f8dd93af21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:26.977769 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da0ae6-c831-40e4-9ab8-f2f8dd93af21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:26.979390 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da0ae6-c831-40e4-9ab8-f2f8dd93af21 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:26.991265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 790/1583] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:26.994645 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da0ae6-c831-40e4-9ab8-f2f8dd93af21 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.005517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 794/1584] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:26 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:27.005759 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da0ae6-c831-40e4-9ab8-f2f8dd93af21 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.005866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47e433-b9a1-4547-95c2-f98cafa25daf admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.062548 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b7a81-df2f-4b63-9874-d7163490ee8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.063189 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b7a81-df2f-4b63-9874-d7163490ee8a admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.063472 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b7a81-df2f-4b63-9874-d7163490ee8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.063747 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b7a81-df2f-4b63-9874-d7163490ee8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.064390 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b7a81-df2f-4b63-9874-d7163490ee8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.066029 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b7a81-df2f-4b63-9874-d7163490ee8a admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.068020 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47e433-b9a1-4547-95c2-f98cafa25daf admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.072629 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09d4c1-b4ce-4f03-bd83-5ebe1ad1faa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.073236 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09d4c1-b4ce-4f03-bd83-5ebe1ad1faa4 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.073511 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09d4c1-b4ce-4f03-bd83-5ebe1ad1faa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.073817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09d4c1-b4ce-4f03-bd83-5ebe1ad1faa4 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.074443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b09d4c1-b4ce-4f03-bd83-5ebe1ad1faa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.076395 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b09d4c1-b4ce-4f03-bd83-5ebe1ad1faa4 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.076563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 791/1585] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:27.077340 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b7a81-df2f-4b63-9874-d7163490ee8a admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.091912 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b7a81-df2f-4b63-9874-d7163490ee8a admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.097193 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620ec91-80c5-49de-832e-58b14eea8e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.097834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620ec91-80c5-49de-832e-58b14eea8e69 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.098119 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620ec91-80c5-49de-832e-58b14eea8e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.098389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620ec91-80c5-49de-832e-58b14eea8e69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.099026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d620ec91-80c5-49de-832e-58b14eea8e69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.100215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 795/1586] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:27 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:27.100646 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d620ec91-80c5-49de-832e-58b14eea8e69 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.104233 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09d4c1-b4ce-4f03-bd83-5ebe1ad1faa4 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.112568 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 792/1587] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:27.113395 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620ec91-80c5-49de-832e-58b14eea8e69 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.115612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09d4c1-b4ce-4f03-bd83-5ebe1ad1faa4 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.120852 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ce09b-f435-40e3-9bcc-04ef06a40906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.121494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ce09b-f435-40e3-9bcc-04ef06a40906 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.121795 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ce09b-f435-40e3-9bcc-04ef06a40906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.122074 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ce09b-f435-40e3-9bcc-04ef06a40906 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.122787 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ce09b-f435-40e3-9bcc-04ef06a40906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.122902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620ec91-80c5-49de-832e-58b14eea8e69 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.124431 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ce09b-f435-40e3-9bcc-04ef06a40906 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.127420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacaa455-3a75-4bad-a99c-dae1020c8691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.127932 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacaa455-3a75-4bad-a99c-dae1020c8691 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.128207 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacaa455-3a75-4bad-a99c-dae1020c8691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.128477 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacaa455-3a75-4bad-a99c-dae1020c8691 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.129101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacaa455-3a75-4bad-a99c-dae1020c8691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.130691 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacaa455-3a75-4bad-a99c-dae1020c8691 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.135174 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 796/1588] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:27.135975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ce09b-f435-40e3-9bcc-04ef06a40906 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.147635 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 793/1589] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:27.147635 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacaa455-3a75-4bad-a99c-dae1020c8691 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.147910 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ce09b-f435-40e3-9bcc-04ef06a40906 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.152193 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b72d16-d2b5-438f-86f5-971325f0b9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.152783 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b72d16-d2b5-438f-86f5-971325f0b9e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.153097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b72d16-d2b5-438f-86f5-971325f0b9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.153371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b72d16-d2b5-438f-86f5-971325f0b9e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.154092 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b72d16-d2b5-438f-86f5-971325f0b9e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.156123 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacaa455-3a75-4bad-a99c-dae1020c8691 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.156244 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b72d16-d2b5-438f-86f5-971325f0b9e8 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.160755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e204139-e81d-4e27-aaa4-96acb7f12c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.161334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e204139-e81d-4e27-aaa4-96acb7f12c6d admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.162081 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e204139-e81d-4e27-aaa4-96acb7f12c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.162081 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e204139-e81d-4e27-aaa4-96acb7f12c6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.162554 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e204139-e81d-4e27-aaa4-96acb7f12c6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.164663 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e204139-e81d-4e27-aaa4-96acb7f12c6d admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.167610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 797/1590] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:27.168572 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b72d16-d2b5-438f-86f5-971325f0b9e8 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.179506 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 794/1591] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:27.179506 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e204139-e81d-4e27-aaa4-96acb7f12c6d admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.182144 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b72d16-d2b5-438f-86f5-971325f0b9e8 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.187824 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf6123-7835-4b76-b3ee-8d12bced18bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.187991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e204139-e81d-4e27-aaa4-96acb7f12c6d admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.188415 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf6123-7835-4b76-b3ee-8d12bced18bd admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.188697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf6123-7835-4b76-b3ee-8d12bced18bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.188998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf6123-7835-4b76-b3ee-8d12bced18bd admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.189621 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf6123-7835-4b76-b3ee-8d12bced18bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.191569 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf6123-7835-4b76-b3ee-8d12bced18bd admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.193167 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb6c55b-78f9-420c-8472-01f02d9a0c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.193864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6c55b-78f9-420c-8472-01f02d9a0c70 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.194191 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6c55b-78f9-420c-8472-01f02d9a0c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.194536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6c55b-78f9-420c-8472-01f02d9a0c70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.195278 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb6c55b-78f9-420c-8472-01f02d9a0c70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.196999 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb6c55b-78f9-420c-8472-01f02d9a0c70 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.209603 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 795/1592] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:27.214104 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb6c55b-78f9-420c-8472-01f02d9a0c70 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.218208 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 798/1593] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:27 2022] POST /identity/v3/users => generated 291 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:27.218208 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf6123-7835-4b76-b3ee-8d12bced18bd admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.224061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb6c55b-78f9-420c-8472-01f02d9a0c70 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.225952 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf6123-7835-4b76-b3ee-8d12bced18bd admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.228760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add1a5b-7db6-48b8-990c-3c10897c18bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.229350 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add1a5b-7db6-48b8-990c-3c10897c18bf admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.229623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add1a5b-7db6-48b8-990c-3c10897c18bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.229922 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add1a5b-7db6-48b8-990c-3c10897c18bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.230549 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add1a5b-7db6-48b8-990c-3c10897c18bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.230658 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c86a87-7743-42ae-bcf0-25ce400e524d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.231163 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c86a87-7743-42ae-bcf0-25ce400e524d admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.231440 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c86a87-7743-42ae-bcf0-25ce400e524d admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.231791 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c86a87-7743-42ae-bcf0-25ce400e524d admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.232524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add1a5b-7db6-48b8-990c-3c10897c18bf admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.232653 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c86a87-7743-42ae-bcf0-25ce400e524d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.234147 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c86a87-7743-42ae-bcf0-25ce400e524d admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.243073 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 796/1594] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:27.243876 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add1a5b-7db6-48b8-990c-3c10897c18bf admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.247108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 799/1595] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:27.247108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c86a87-7743-42ae-bcf0-25ce400e524d admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.256381 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c86a87-7743-42ae-bcf0-25ce400e524d admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.257160 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add1a5b-7db6-48b8-990c-3c10897c18bf admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.261045 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8f3b5e-c6e3-4647-b097-ea99e7291105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.261629 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f3b5e-c6e3-4647-b097-ea99e7291105 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.261934 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f3b5e-c6e3-4647-b097-ea99e7291105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.262051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd93ea-07f3-425f-9f82-3b86104c4359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.262204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f3b5e-c6e3-4647-b097-ea99e7291105 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.262578 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd93ea-07f3-425f-9f82-3b86104c4359 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.262833 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8f3b5e-c6e3-4647-b097-ea99e7291105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.263088 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd93ea-07f3-425f-9f82-3b86104c4359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.263294 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd93ea-07f3-425f-9f82-3b86104c4359 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.264533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd93ea-07f3-425f-9f82-3b86104c4359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.265003 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8f3b5e-c6e3-4647-b097-ea99e7291105 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.266163 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd93ea-07f3-425f-9f82-3b86104c4359 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.276366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 800/1596] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:27.276514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 797/1597] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:27.276972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd93ea-07f3-425f-9f82-3b86104c4359 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.280333 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8f3b5e-c6e3-4647-b097-ea99e7291105 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.286693 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd93ea-07f3-425f-9f82-3b86104c4359 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.290159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8f3b5e-c6e3-4647-b097-ea99e7291105 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.291509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c58fa-eb3f-460d-9d94-4a4e35bb5434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.292226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c58fa-eb3f-460d-9d94-4a4e35bb5434 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.292573 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c58fa-eb3f-460d-9d94-4a4e35bb5434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.292946 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c58fa-eb3f-460d-9d94-4a4e35bb5434 admin admin] PATH_INFO: `/v3/projects/b644d63d813746bfa8dc52f42973aba0/users/3622f140137340e985d256a64fc360e6/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.294944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471ad59-4d52-48b4-a14c-4c78304c529b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.295524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471ad59-4d52-48b4-a14c-4c78304c529b admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.295800 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471ad59-4d52-48b4-a14c-4c78304c529b admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.296072 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471ad59-4d52-48b4-a14c-4c78304c529b admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.296706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471ad59-4d52-48b4-a14c-4c78304c529b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.298410 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471ad59-4d52-48b4-a14c-4c78304c529b admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.308132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c58fa-eb3f-460d-9d94-4a4e35bb5434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b644d63d813746bfa8dc52f42973aba0, user_id=3622f140137340e985d256a64fc360e6, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.308353 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 801/1598] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:27.312652 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471ad59-4d52-48b4-a14c-4c78304c529b admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.312776 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c58fa-eb3f-460d-9d94-4a4e35bb5434 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.319933 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471ad59-4d52-48b4-a14c-4c78304c529b admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.324676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 798/1599] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:27 2022] PUT /identity/v3/projects/b644d63d813746bfa8dc52f42973aba0/users/3622f140137340e985d256a64fc360e6/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:27.326569 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c58fa-eb3f-460d-9d94-4a4e35bb5434 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.326961 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:27.375150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:27.382247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91127d97-d135-4988-afd3-30978711fb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.382853 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91127d97-d135-4988-afd3-30978711fb62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.383130 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91127d97-d135-4988-afd3-30978711fb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.383401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91127d97-d135-4988-afd3-30978711fb62 admin admin] PATH_INFO: `/v3/projects/0ef97e4835ab48d8935356ba4afa5918/users/9afe8e3912cf489a994e4f13e3914c8e/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.391915 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c58fa-eb3f-460d-9d94-4a4e35bb5434 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.396885 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ea6df-f9cc-4c79-94e6-a6a6a0fbb1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.397490 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ea6df-f9cc-4c79-94e6-a6a6a0fbb1c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.397816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ea6df-f9cc-4c79-94e6-a6a6a0fbb1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.398089 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ea6df-f9cc-4c79-94e6-a6a6a0fbb1c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.398724 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ea6df-f9cc-4c79-94e6-a6a6a0fbb1c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.399965 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91127d97-d135-4988-afd3-30978711fb62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ef97e4835ab48d8935356ba4afa5918, user_id=9afe8e3912cf489a994e4f13e3914c8e, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.400379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ea6df-f9cc-4c79-94e6-a6a6a0fbb1c4 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.403165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91127d97-d135-4988-afd3-30978711fb62 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.414002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 799/1600] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:27.414002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ea6df-f9cc-4c79-94e6-a6a6a0fbb1c4 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.418984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 802/1601] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:27 2022] PUT /identity/v3/projects/0ef97e4835ab48d8935356ba4afa5918/users/9afe8e3912cf489a994e4f13e3914c8e/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:27.421888 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ea6df-f9cc-4c79-94e6-a6a6a0fbb1c4 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.422643 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91127d97-d135-4988-afd3-30978711fb62 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.481197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c7ddc-55fe-420f-a757-2483feeca4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.482475 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c7ddc-55fe-420f-a757-2483feeca4a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.482826 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c7ddc-55fe-420f-a757-2483feeca4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.483105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c7ddc-55fe-420f-a757-2483feeca4a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.483742 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c7ddc-55fe-420f-a757-2483feeca4a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.485348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c7ddc-55fe-420f-a757-2483feeca4a5 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.487775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91127d97-d135-4988-afd3-30978711fb62 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.492678 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6f5b4-9170-4f20-b40a-8bf297c35a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.493339 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6f5b4-9170-4f20-b40a-8bf297c35a3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.493618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6f5b4-9170-4f20-b40a-8bf297c35a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.493923 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6f5b4-9170-4f20-b40a-8bf297c35a3d admin admin] PATH_INFO: `/v3/projects/bbb0263903e14938b44d9a6b430d84ce/users/e75b537d44524de192282b10825293f9/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.499240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 800/1602] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:27.499240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c7ddc-55fe-420f-a757-2483feeca4a5 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.506366 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c7ddc-55fe-420f-a757-2483feeca4a5 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.509529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d6f5b4-9170-4f20-b40a-8bf297c35a3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbb0263903e14938b44d9a6b430d84ce, user_id=e75b537d44524de192282b10825293f9, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.511354 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfca48-8407-4eb3-bc27-db58d4614e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.512001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfca48-8407-4eb3-bc27-db58d4614e04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.512282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfca48-8407-4eb3-bc27-db58d4614e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.512391 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d6f5b4-9170-4f20-b40a-8bf297c35a3d admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.512689 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfca48-8407-4eb3-bc27-db58d4614e04 admin admin] PATH_INFO: `/v3/projects/fb059805c019400ea97c73dda8404254/users/897041db45a84d3fbb1b680c83efafdf/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.526124 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 803/1603] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:27 2022] PUT /identity/v3/projects/bbb0263903e14938b44d9a6b430d84ce/users/e75b537d44524de192282b10825293f9/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:27.531018 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6f5b4-9170-4f20-b40a-8bf297c35a3d admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.531124 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfca48-8407-4eb3-bc27-db58d4614e04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb059805c019400ea97c73dda8404254, user_id=897041db45a84d3fbb1b680c83efafdf, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.531644 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfca48-8407-4eb3-bc27-db58d4614e04 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.546275 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 801/1604] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:27 2022] PUT /identity/v3/projects/fb059805c019400ea97c73dda8404254/users/897041db45a84d3fbb1b680c83efafdf/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:27.547163 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfca48-8407-4eb3-bc27-db58d4614e04 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.598722 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6f5b4-9170-4f20-b40a-8bf297c35a3d admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.618038 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfca48-8407-4eb3-bc27-db58d4614e04 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.618618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:27.623187 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60603fa-89d4-4aca-9aed-449f8c588a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.623854 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60603fa-89d4-4aca-9aed-449f8c588a17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.624132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60603fa-89d4-4aca-9aed-449f8c588a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.624402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60603fa-89d4-4aca-9aed-449f8c588a17 admin admin] PATH_INFO: `/v3/projects/6fb06a5fd3a3482d991c542ccc993e88/users/04c2aae2da284df29243b99d33725ddd/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.640663 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60603fa-89d4-4aca-9aed-449f8c588a17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fb06a5fd3a3482d991c542ccc993e88, user_id=04c2aae2da284df29243b99d33725ddd, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.643319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60603fa-89d4-4aca-9aed-449f8c588a17 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.657840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 802/1605] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:27 2022] PUT /identity/v3/projects/6fb06a5fd3a3482d991c542ccc993e88/users/04c2aae2da284df29243b99d33725ddd/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:27.659310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746dc93-58a4-4331-8312-b29c1312700d None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.659874 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746dc93-58a4-4331-8312-b29c1312700d None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.660287 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3a6f5d-2245-422b-818f-7d3fbb15dc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.660480 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746dc93-58a4-4331-8312-b29c1312700d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.660953 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a6f5d-2245-422b-818f-7d3fbb15dc66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.661320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a6f5d-2245-422b-818f-7d3fbb15dc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.661602 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a6f5d-2245-422b-818f-7d3fbb15dc66 admin admin] PATH_INFO: `/v3/projects/bcff03cfe90e4d1bb5f9bf1564edafa6/users/2420a84d76e845c68bd14852b1a910a9/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.677568 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3a6f5d-2245-422b-818f-7d3fbb15dc66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcff03cfe90e4d1bb5f9bf1564edafa6, user_id=2420a84d76e845c68bd14852b1a910a9, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.680291 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3a6f5d-2245-422b-818f-7d3fbb15dc66 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.690614 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-d746dc93-58a4-4331-8312-b29c1312700d None None] MFA Rules not processed for user `3622f140137340e985d256a64fc360e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:27.691858 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d746dc93-58a4-4331-8312-b29c1312700d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:27.693262 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 804/1606] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:27 2022] PUT /identity/v3/projects/bcff03cfe90e4d1bb5f9bf1564edafa6/users/2420a84d76e845c68bd14852b1a910a9/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:27.694031 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3a6f5d-2245-422b-818f-7d3fbb15dc66 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.720455 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:27.759531 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3a6f5d-2245-422b-818f-7d3fbb15dc66 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.762390 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 803/1607] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:27.764319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93ea7c-d321-4239-9fcb-a0011f80c426 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.764767 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93ea7c-d321-4239-9fcb-a0011f80c426 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.765066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba79d8-a87a-47c8-b4f0-cd43ef60abf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.765395 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93ea7c-d321-4239-9fcb-a0011f80c426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.765752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba79d8-a87a-47c8-b4f0-cd43ef60abf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.766035 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba79d8-a87a-47c8-b4f0-cd43ef60abf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.766305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba79d8-a87a-47c8-b4f0-cd43ef60abf9 admin admin] PATH_INFO: `/v3/projects/5926f39634e84952a23ff4375cbfc22f/users/d8d0bc6d996f44279fd6bc879a88e52d/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.782205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba79d8-a87a-47c8-b4f0-cd43ef60abf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5926f39634e84952a23ff4375cbfc22f, user_id=d8d0bc6d996f44279fd6bc879a88e52d, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.784841 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba79d8-a87a-47c8-b4f0-cd43ef60abf9 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.795024 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-8d93ea7c-d321-4239-9fcb-a0011f80c426 None None] MFA Rules not processed for user `9afe8e3912cf489a994e4f13e3914c8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:27.796358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-8d93ea7c-d321-4239-9fcb-a0011f80c426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:27.803058 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 805/1608] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:27 2022] PUT /identity/v3/projects/5926f39634e84952a23ff4375cbfc22f/users/d8d0bc6d996f44279fd6bc879a88e52d/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:27.804050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba79d8-a87a-47c8-b4f0-cd43ef60abf9 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:27.868807 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 804/1609] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:27 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:27.870351 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d8fd1-41b9-4daa-a9e6-951289e5f768 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.870802 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d8fd1-41b9-4daa-a9e6-951289e5f768 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.871234 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d8fd1-41b9-4daa-a9e6-951289e5f768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.874018 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba79d8-a87a-47c8-b4f0-cd43ef60abf9 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:27.878687 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd59631-5fcc-425b-9999-1098cfbc7e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:27.879344 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd59631-5fcc-425b-9999-1098cfbc7e40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.879624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd59631-5fcc-425b-9999-1098cfbc7e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.879893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd59631-5fcc-425b-9999-1098cfbc7e40 admin admin] PATH_INFO: `/v3/projects/cbb0922aa97f4f39bb7a426a0a94ffb1/users/8a55334adb294937abaf0c2bef36be3e/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.890467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-6e4d8fd1-41b9-4daa-a9e6-951289e5f768 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:27.891742 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-6e4d8fd1-41b9-4daa-a9e6-951289e5f768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:27.896660 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd59631-5fcc-425b-9999-1098cfbc7e40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbb0922aa97f4f39bb7a426a0a94ffb1, user_id=8a55334adb294937abaf0c2bef36be3e, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:27.899395 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd59631-5fcc-425b-9999-1098cfbc7e40 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:27.899789 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 805/1610] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:27.901059 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1228ff7-98e2-445b-a5c6-801a6cbacb18 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.901346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1228ff7-98e2-445b-a5c6-801a6cbacb18 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.901619 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1228ff7-98e2-445b-a5c6-801a6cbacb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.919629 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 806/1611] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:27 2022] PUT /identity/v3/projects/cbb0922aa97f4f39bb7a426a0a94ffb1/users/8a55334adb294937abaf0c2bef36be3e/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:27.929217 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4045f-4159-4c3f-949b-269e3cdb2e49 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:27.929217 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4045f-4159-4c3f-949b-269e3cdb2e49 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:27.929217 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4045f-4159-4c3f-949b-269e3cdb2e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:27.939648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-b1228ff7-98e2-445b-a5c6-801a6cbacb18 None None] MFA Rules not processed for user `897041db45a84d3fbb1b680c83efafdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:27.940848 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-b1228ff7-98e2-445b-a5c6-801a6cbacb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:27.943154 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-3ba4045f-4159-4c3f-949b-269e3cdb2e49 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:27.944397 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-3ba4045f-4159-4c3f-949b-269e3cdb2e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:27.949576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:28.006556 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 806/1612] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:28.007808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 807/1613] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:28.008681 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4045f-4159-4c3f-949b-269e3cdb2e49 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:28.010910 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6e962-e96e-416a-8ade-fb5357aa74e5 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:28.011250 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6e962-e96e-416a-8ade-fb5357aa74e5 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:28.011498 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6e962-e96e-416a-8ade-fb5357aa74e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:28.019648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4045f-4159-4c3f-949b-269e3cdb2e49 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:28.024924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0db81-2bfc-4c67-8bd7-5e416db6caee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:28.025586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0db81-2bfc-4c67-8bd7-5e416db6caee admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:28.025914 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0db81-2bfc-4c67-8bd7-5e416db6caee admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:28.026186 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0db81-2bfc-4c67-8bd7-5e416db6caee admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:28.026815 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d0db81-2bfc-4c67-8bd7-5e416db6caee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:28.028751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d0db81-2bfc-4c67-8bd7-5e416db6caee admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:28.031755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-49f6e962-e96e-416a-8ade-fb5357aa74e5 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:28.032947 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-49f6e962-e96e-416a-8ade-fb5357aa74e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:28.041549 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 807/1614] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:28.042535 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6e962-e96e-416a-8ade-fb5357aa74e5 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:28.055203 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6e962-e96e-416a-8ade-fb5357aa74e5 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:28.057926 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 808/1615] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:28 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:28.058890 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0db81-2bfc-4c67-8bd7-5e416db6caee admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:28.060388 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfe59c-aeef-4147-938b-e5353385a1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:28.061081 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfe59c-aeef-4147-938b-e5353385a1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:28.061473 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfe59c-aeef-4147-938b-e5353385a1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:28.061726 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfe59c-aeef-4147-938b-e5353385a1b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:28.062392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dfe59c-aeef-4147-938b-e5353385a1b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:28.063994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dfe59c-aeef-4147-938b-e5353385a1b0 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:28.074943 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 808/1616] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:28.079230 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfe59c-aeef-4147-938b-e5353385a1b0 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:28.104853 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:28.131352 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0db81-2bfc-4c67-8bd7-5e416db6caee admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:28.135591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfe59c-aeef-4147-938b-e5353385a1b0 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:28.136000 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ef6ca-c183-46a9-96d3-806294a9702b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:28.136601 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ef6ca-c183-46a9-96d3-806294a9702b admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:28.136877 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ef6ca-c183-46a9-96d3-806294a9702b admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:28.137149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ef6ca-c183-46a9-96d3-806294a9702b admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:28.137806 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ef6ca-c183-46a9-96d3-806294a9702b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:28.140261 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ef6ca-c183-46a9-96d3-806294a9702b admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:28.167692 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 809/1617] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:28 2022] POST /identity/v3/projects => generated 403 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:28.168821 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82779b72-2cab-400c-9538-f11830a28912 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:28.169111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82779b72-2cab-400c-9538-f11830a28912 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:28.169435 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82779b72-2cab-400c-9538-f11830a28912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:28.188818 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b62b27-6e26-404c-b4ee-7a742b5826ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:28.189588 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b62b27-6e26-404c-b4ee-7a742b5826ed service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:28.190646 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b62b27-6e26-404c-b4ee-7a742b5826ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:28.190932 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b62b27-6e26-404c-b4ee-7a742b5826ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:28.197032 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-82779b72-2cab-400c-9538-f11830a28912 None None] MFA Rules not processed for user `8a55334adb294937abaf0c2bef36be3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:28.198256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-82779b72-2cab-400c-9538-f11830a28912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:28.200092 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b62b27-6e26-404c-b4ee-7a742b5826ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:28.202111 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b62b27-6e26-404c-b4ee-7a742b5826ed service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:28.211876 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 809/1618] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:28.212419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b62b27-6e26-404c-b4ee-7a742b5826ed service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:28.222366 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b62b27-6e26-404c-b4ee-7a742b5826ed service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:28.275376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 810/1619] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:28.278832 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82779b72-2cab-400c-9538-f11830a28912 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:28.291495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:28.308653 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e4976-0b8b-49d4-b042-4760b090b621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:28.310598 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e4976-0b8b-49d4-b042-4760b090b621 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:28.310887 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e4976-0b8b-49d4-b042-4760b090b621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:28.311167 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e4976-0b8b-49d4-b042-4760b090b621 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:28.311811 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e4976-0b8b-49d4-b042-4760b090b621 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:28.314672 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e4976-0b8b-49d4-b042-4760b090b621 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:28.348623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 810/1620] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:28 2022] POST /identity/v3/projects => generated 367 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:28.349635 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e4976-0b8b-49d4-b042-4760b090b621 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:28.359163 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82779b72-2cab-400c-9538-f11830a28912 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:28.414769 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e4976-0b8b-49d4-b042-4760b090b621 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:28.419975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c097a-bb1b-4967-992c-eb427c253ec0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:28.420507 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c097a-bb1b-4967-992c-eb427c253ec0 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:28.420937 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c097a-bb1b-4967-992c-eb427c253ec0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:28.421212 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c097a-bb1b-4967-992c-eb427c253ec0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:28.421743 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298d392-e7e4-466d-b1c2-9aed4252fb1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:28.422368 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298d392-e7e4-466d-b1c2-9aed4252fb1e service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:28.422655 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298d392-e7e4-466d-b1c2-9aed4252fb1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:28.422913 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298d392-e7e4-466d-b1c2-9aed4252fb1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:28.446129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:28.499936 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298d392-e7e4-466d-b1c2-9aed4252fb1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:28.501567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298d392-e7e4-466d-b1c2-9aed4252fb1e service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:28.505020 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0c097a-bb1b-4967-992c-eb427c253ec0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:28.507368 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0c097a-bb1b-4967-992c-eb427c253ec0 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:28.516858 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 811/1621] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:28.517818 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298d392-e7e4-466d-b1c2-9aed4252fb1e service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:28.519979 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 811/1622] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:28.520951 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c097a-bb1b-4967-992c-eb427c253ec0 service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:28.529303 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298d392-e7e4-466d-b1c2-9aed4252fb1e service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:28.530540 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c097a-bb1b-4967-992c-eb427c253ec0 service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:28.534230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67113e3d-96ae-44b8-91ff-e8b7dbb3b961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:28.535010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67113e3d-96ae-44b8-91ff-e8b7dbb3b961 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:28.535260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598078f-f66f-4a11-aab0-c2ddb42e9cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:28.535782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67113e3d-96ae-44b8-91ff-e8b7dbb3b961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:28.535909 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598078f-f66f-4a11-aab0-c2ddb42e9cb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:28.536070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67113e3d-96ae-44b8-91ff-e8b7dbb3b961 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:28.536176 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598078f-f66f-4a11-aab0-c2ddb42e9cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:28.536426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598078f-f66f-4a11-aab0-c2ddb42e9cb5 admin admin] PATH_INFO: `/v3/projects/5926f39634e84952a23ff4375cbfc22f/users/d8d0bc6d996f44279fd6bc879a88e52d/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:28.536745 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67113e3d-96ae-44b8-91ff-e8b7dbb3b961 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:28.538434 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598078f-f66f-4a11-aab0-c2ddb42e9cb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5926f39634e84952a23ff4375cbfc22f, user_id=d8d0bc6d996f44279fd6bc879a88e52d, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:28.538906 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67113e3d-96ae-44b8-91ff-e8b7dbb3b961 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:28.540981 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598078f-f66f-4a11-aab0-c2ddb42e9cb5 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:28.556241 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 812/1623] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:28 2022] PUT /identity/v3/projects/5926f39634e84952a23ff4375cbfc22f/users/d8d0bc6d996f44279fd6bc879a88e52d/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:28.557694 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598078f-f66f-4a11-aab0-c2ddb42e9cb5 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:28.572950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 812/1624] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:28 2022] POST /identity/v3/users => generated 289 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:28.574519 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746bb72-3bda-4f62-8565-62ae86ab46a8 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:28.576501 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746bb72-3bda-4f62-8565-62ae86ab46a8 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:28.576798 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746bb72-3bda-4f62-8565-62ae86ab46a8 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:28.578492 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 813/1625] 10.222.0.73 () {54 vars in 962 bytes} [Mon Aug 29 20:12:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:12:28.583613 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746bb72-3bda-4f62-8565-62ae86ab46a8 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:28.646025 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598078f-f66f-4a11-aab0-c2ddb42e9cb5 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:28.651155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:28.651724 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8eac8-eb59-4303-8bdd-3464226077a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:28.652343 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8eac8-eb59-4303-8bdd-3464226077a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:28.652621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8eac8-eb59-4303-8bdd-3464226077a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:28.653326 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8eac8-eb59-4303-8bdd-3464226077a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:28.655095 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e8eac8-eb59-4303-8bdd-3464226077a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:28.655605 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e8eac8-eb59-4303-8bdd-3464226077a5 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:28.666720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746bb72-3bda-4f62-8565-62ae86ab46a8 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:28.675007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81031d9c-b35c-44cb-ae86-5b6262a1a472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:28.675878 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81031d9c-b35c-44cb-ae86-5b6262a1a472 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:28.676350 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81031d9c-b35c-44cb-ae86-5b6262a1a472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:28.676741 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81031d9c-b35c-44cb-ae86-5b6262a1a472 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:28.677542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81031d9c-b35c-44cb-ae86-5b6262a1a472 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:28.679751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81031d9c-b35c-44cb-ae86-5b6262a1a472 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:28.681112 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 813/1626] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:28 2022] POST /identity/v3/users => generated 308 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:28.682387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995f3a6-9e7c-4c30-9416-9dba27620980 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:28.683837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995f3a6-9e7c-4c30-9416-9dba27620980 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:28.683837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995f3a6-9e7c-4c30-9416-9dba27620980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:28.710245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 814/1627] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:28 2022] POST /identity/v3/users => generated 290 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:28.711430 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81031d9c-b35c-44cb-ae86-5b6262a1a472 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:28.722433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-1995f3a6-9e7c-4c30-9416-9dba27620980 None None] MFA Rules not processed for user `d8d0bc6d996f44279fd6bc879a88e52d`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:28.724258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-1995f3a6-9e7c-4c30-9416-9dba27620980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:28.725074 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81031d9c-b35c-44cb-ae86-5b6262a1a472 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:28.731459 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da6e12-eff8-4e22-bc87-b42c0c69eb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:28.732630 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da6e12-eff8-4e22-bc87-b42c0c69eb1e admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:28.732955 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da6e12-eff8-4e22-bc87-b42c0c69eb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:28.733225 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da6e12-eff8-4e22-bc87-b42c0c69eb1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:28.734159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da6e12-eff8-4e22-bc87-b42c0c69eb1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:28.739561 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da6e12-eff8-4e22-bc87-b42c0c69eb1e admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:28.752048 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 815/1628] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:28.753514 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e5905-7526-4025-a212-cebbf38f3004 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:28.754343 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e5905-7526-4025-a212-cebbf38f3004 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:28.754343 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e5905-7526-4025-a212-cebbf38f3004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:28.825550 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-b85e5905-7526-4025-a212-cebbf38f3004 None None] MFA Rules not processed for user `95cf77a92a174ff8b0503baf5c7e63be`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:28.827466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-b85e5905-7526-4025-a212-cebbf38f3004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:28.836238 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 814/1629] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:28 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:28.844134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995f3a6-9e7c-4c30-9416-9dba27620980 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:28.863187 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995f3a6-9e7c-4c30-9416-9dba27620980 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:28.870026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870e19a-9145-4b62-9484-c09cd1031893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:28.872319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870e19a-9145-4b62-9484-c09cd1031893 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:28.872678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870e19a-9145-4b62-9484-c09cd1031893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:28.873172 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870e19a-9145-4b62-9484-c09cd1031893 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:28.874298 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870e19a-9145-4b62-9484-c09cd1031893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:28.876381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870e19a-9145-4b62-9484-c09cd1031893 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:28.906046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 815/1630] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:28.907215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870e19a-9145-4b62-9484-c09cd1031893 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:28.922908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870e19a-9145-4b62-9484-c09cd1031893 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:28.930655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d62c5-9231-48d9-ae21-ba69170a02f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:28.931416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d62c5-9231-48d9-ae21-ba69170a02f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:28.931899 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d62c5-9231-48d9-ae21-ba69170a02f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:28.932317 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d62c5-9231-48d9-ae21-ba69170a02f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:28.933155 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d62c5-9231-48d9-ae21-ba69170a02f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:28.935543 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d62c5-9231-48d9-ae21-ba69170a02f1 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:28.937468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:28.952976 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 816/1631] 10.222.0.73 () {58 vars in 1059 bytes} [Mon Aug 29 20:12:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:28.953212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e5905-7526-4025-a212-cebbf38f3004 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:28.953817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 816/1632] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:28.955012 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d62c5-9231-48d9-ae21-ba69170a02f1 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:28.970445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e5905-7526-4025-a212-cebbf38f3004 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:28.977302 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e4970-94d7-4a42-a930-36ea0b8a0fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:28.978072 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e4970-94d7-4a42-a930-36ea0b8a0fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:28.978363 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e4970-94d7-4a42-a930-36ea0b8a0fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:28.978645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e4970-94d7-4a42-a930-36ea0b8a0fd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:28.979455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e4970-94d7-4a42-a930-36ea0b8a0fd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:28.981297 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e4970-94d7-4a42-a930-36ea0b8a0fd3 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:29.004082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 817/1633] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:29.004082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e4970-94d7-4a42-a930-36ea0b8a0fd3 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:29.022131 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e4970-94d7-4a42-a930-36ea0b8a0fd3 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:29.035817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333e6bf-1a06-4c22-aca0-77779a7691b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:29.036648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333e6bf-1a06-4c22-aca0-77779a7691b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:29.037142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333e6bf-1a06-4c22-aca0-77779a7691b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:29.037830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333e6bf-1a06-4c22-aca0-77779a7691b6 admin admin] PATH_INFO: `/v3/projects/3e52facf2a124b95928f1145a70ea2ae/users/3280ccc382934a77b1f3820edcc11830/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:29.061617 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d62c5-9231-48d9-ae21-ba69170a02f1 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:29.066018 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333e6bf-1a06-4c22-aca0-77779a7691b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e52facf2a124b95928f1145a70ea2ae, user_id=3280ccc382934a77b1f3820edcc11830, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:29.069040 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333e6bf-1a06-4c22-aca0-77779a7691b6 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:29.070690 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a621ce-08c7-4f35-b15f-cb0108069480 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:29.071637 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a621ce-08c7-4f35-b15f-cb0108069480 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:29.072172 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a621ce-08c7-4f35-b15f-cb0108069480 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:29.072907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a621ce-08c7-4f35-b15f-cb0108069480 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:29.091167 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 818/1634] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:29 2022] PUT /identity/v3/projects/3e52facf2a124b95928f1145a70ea2ae/users/3280ccc382934a77b1f3820edcc11830/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:29.093597 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333e6bf-1a06-4c22-aca0-77779a7691b6 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:29.100296 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a621ce-08c7-4f35-b15f-cb0108069480 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:29.103175 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a621ce-08c7-4f35-b15f-cb0108069480 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:29.118026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 817/1635] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:28 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:29.119864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a621ce-08c7-4f35-b15f-cb0108069480 service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:29.133276 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a621ce-08c7-4f35-b15f-cb0108069480 service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:29.197333 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333e6bf-1a06-4c22-aca0-77779a7691b6 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:29.212720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfebe55-2f7f-497c-9d9b-e4ddea165df7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:29.213563 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfebe55-2f7f-497c-9d9b-e4ddea165df7 service nova] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:29.214093 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfebe55-2f7f-497c-9d9b-e4ddea165df7 service nova] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:29.214606 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfebe55-2f7f-497c-9d9b-e4ddea165df7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:29.218641 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d95378-df3c-4328-961c-53f54962ef80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:29.219430 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d95378-df3c-4328-961c-53f54962ef80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:29.219779 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d95378-df3c-4328-961c-53f54962ef80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:29.220132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d95378-df3c-4328-961c-53f54962ef80 admin admin] PATH_INFO: `/v3/projects/8bc0b25f4adb4535828370a0cac25577/users/058e7536fa7d4c6abceba8c21079d01a/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:29.232797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:29.259354 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d95378-df3c-4328-961c-53f54962ef80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bc0b25f4adb4535828370a0cac25577, user_id=058e7536fa7d4c6abceba8c21079d01a, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:29.265672 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d95378-df3c-4328-961c-53f54962ef80 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:29.297791 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 818/1636] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:29 2022] PUT /identity/v3/projects/8bc0b25f4adb4535828370a0cac25577/users/058e7536fa7d4c6abceba8c21079d01a/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:29.297791 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d95378-df3c-4328-961c-53f54962ef80 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:29.317512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfebe55-2f7f-497c-9d9b-e4ddea165df7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:29.320425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfebe55-2f7f-497c-9d9b-e4ddea165df7 service nova] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:29.337996 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 819/1637] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:29.339224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfebe55-2f7f-497c-9d9b-e4ddea165df7 service nova] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:29.371111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfebe55-2f7f-497c-9d9b-e4ddea165df7 service nova] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:29.458644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a56b9-695a-451a-8ef6-c870bbd76842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:29.459270 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a56b9-695a-451a-8ef6-c870bbd76842 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:29.459915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a56b9-695a-451a-8ef6-c870bbd76842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:29.460198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a56b9-695a-451a-8ef6-c870bbd76842 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:29.460851 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a56b9-695a-451a-8ef6-c870bbd76842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:29.462916 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a56b9-695a-451a-8ef6-c870bbd76842 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:29.474609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d95378-df3c-4328-961c-53f54962ef80 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:29.479983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 820/1638] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:29.481250 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a56b9-695a-451a-8ef6-c870bbd76842 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:29.498203 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a56b9-695a-451a-8ef6-c870bbd76842 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:29.501697 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df88244-e029-426f-ac08-4450a17a53c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:29.502560 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df88244-e029-426f-ac08-4450a17a53c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:29.502849 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df88244-e029-426f-ac08-4450a17a53c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:29.503130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df88244-e029-426f-ac08-4450a17a53c2 admin admin] PATH_INFO: `/v3/projects/7dfda4531b664d18afd50da65858bacb/users/f30daa72513644bba09bf11ffaad3c87/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:29.510676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146714b-9aba-43d6-bf4c-7caf124c79db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:29.511536 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146714b-9aba-43d6-bf4c-7caf124c79db admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:29.511961 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146714b-9aba-43d6-bf4c-7caf124c79db admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:29.512184 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146714b-9aba-43d6-bf4c-7caf124c79db admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:29.513041 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3146714b-9aba-43d6-bf4c-7caf124c79db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:29.515946 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3146714b-9aba-43d6-bf4c-7caf124c79db admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:29.531318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 821/1639] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:29.532278 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146714b-9aba-43d6-bf4c-7caf124c79db admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:29.548392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df88244-e029-426f-ac08-4450a17a53c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dfda4531b664d18afd50da65858bacb, user_id=f30daa72513644bba09bf11ffaad3c87, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:29.549960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146714b-9aba-43d6-bf4c-7caf124c79db admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:29.555112 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7fce80-e226-4822-9ec2-4723cd802b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:29.556168 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fce80-e226-4822-9ec2-4723cd802b9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:29.556454 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fce80-e226-4822-9ec2-4723cd802b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:29.556729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fce80-e226-4822-9ec2-4723cd802b9d admin admin] PATH_INFO: `/v3/projects/3e52facf2a124b95928f1145a70ea2ae/users/3280ccc382934a77b1f3820edcc11830/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:29.558630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df88244-e029-426f-ac08-4450a17a53c2 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:29.558904 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7fce80-e226-4822-9ec2-4723cd802b9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e52facf2a124b95928f1145a70ea2ae, user_id=3280ccc382934a77b1f3820edcc11830, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:29.562832 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7fce80-e226-4822-9ec2-4723cd802b9d admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:29.588280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 822/1640] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:29 2022] PUT /identity/v3/projects/3e52facf2a124b95928f1145a70ea2ae/users/3280ccc382934a77b1f3820edcc11830/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:29.589944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7fce80-e226-4822-9ec2-4723cd802b9d admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:29.602339 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 819/1641] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:29 2022] PUT /identity/v3/projects/7dfda4531b664d18afd50da65858bacb/users/f30daa72513644bba09bf11ffaad3c87/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:29.607104 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf609b-0fe5-4521-a0cd-ddb2187f29b2 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:29.607397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf609b-0fe5-4521-a0cd-ddb2187f29b2 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:29.607683 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf609b-0fe5-4521-a0cd-ddb2187f29b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:29.637557 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:29.688240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-5fcf609b-0fe5-4521-a0cd-ddb2187f29b2 None None] MFA Rules not processed for user `3280ccc382934a77b1f3820edcc11830`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:29.692255 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-5fcf609b-0fe5-4521-a0cd-ddb2187f29b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:29.704669 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7fce80-e226-4822-9ec2-4723cd802b9d admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:29.758585 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:29.820130 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f01bf-a55e-4899-bc35-2676c935c1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:29.820958 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f01bf-a55e-4899-bc35-2676c935c1c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:29.821245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f01bf-a55e-4899-bc35-2676c935c1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:29.821521 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f01bf-a55e-4899-bc35-2676c935c1c4 admin admin] PATH_INFO: `/v3/projects/8bc0b25f4adb4535828370a0cac25577/users/058e7536fa7d4c6abceba8c21079d01a/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:29.825329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f01bf-a55e-4899-bc35-2676c935c1c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bc0b25f4adb4535828370a0cac25577, user_id=058e7536fa7d4c6abceba8c21079d01a, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:29.830578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f01bf-a55e-4899-bc35-2676c935c1c4 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:29.851883 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 823/1642] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:29 2022] PUT /identity/v3/projects/8bc0b25f4adb4535828370a0cac25577/users/058e7536fa7d4c6abceba8c21079d01a/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:29.851883 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c24968-9e8a-4319-93cc-797923e36863 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:29.851883 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c24968-9e8a-4319-93cc-797923e36863 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:29.851883 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c24968-9e8a-4319-93cc-797923e36863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:29.887290 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-39c24968-9e8a-4319-93cc-797923e36863 None None] MFA Rules not processed for user `f30daa72513644bba09bf11ffaad3c87`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:29.892715 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-39c24968-9e8a-4319-93cc-797923e36863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:29.901304 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 820/1643] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:29 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:29.911836 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a1267-5f55-4062-b731-e72c6cf8bd4b None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:29.912762 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a1267-5f55-4062-b731-e72c6cf8bd4b None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:29.913737 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a1267-5f55-4062-b731-e72c6cf8bd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:29.968121 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-665a1267-5f55-4062-b731-e72c6cf8bd4b None None] MFA Rules not processed for user `058e7536fa7d4c6abceba8c21079d01a`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:29.971463 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-665a1267-5f55-4062-b731-e72c6cf8bd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:30.000718 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 824/1644] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:30.001813 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c24968-9e8a-4319-93cc-797923e36863 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:30.103344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 821/1645] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:30.114166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c24968-9e8a-4319-93cc-797923e36863 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:30.118950 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a1267-5f55-4062-b731-e72c6cf8bd4b None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:30.120547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730f0c6-dccd-41f9-9f2f-2af65b6617ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:30.121173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730f0c6-dccd-41f9-9f2f-2af65b6617ee service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:30.121452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730f0c6-dccd-41f9-9f2f-2af65b6617ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:30.121739 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730f0c6-dccd-41f9-9f2f-2af65b6617ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:30.260658 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a1267-5f55-4062-b731-e72c6cf8bd4b None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:30.268374 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730f0c6-dccd-41f9-9f2f-2af65b6617ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:30.273453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730f0c6-dccd-41f9-9f2f-2af65b6617ee service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:30.274440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103fc67-5e3b-456d-8572-587aaa032282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:30.275345 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103fc67-5e3b-456d-8572-587aaa032282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:30.275868 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103fc67-5e3b-456d-8572-587aaa032282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:30.277139 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103fc67-5e3b-456d-8572-587aaa032282 admin admin] PATH_INFO: `/v3/users/103fab04912b42a989f6fa75aa5e4e33` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:30.284739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103fc67-5e3b-456d-8572-587aaa032282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=103fab04912b42a989f6fa75aa5e4e33)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:30.287159 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103fc67-5e3b-456d-8572-587aaa032282 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:30.306814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 825/1646] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:30 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:30.308190 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730f0c6-dccd-41f9-9f2f-2af65b6617ee service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:30.333325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:30.338447 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:30.344168 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730f0c6-dccd-41f9-9f2f-2af65b6617ee service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:30.362924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b57b42-4a5a-4b86-b127-76ae389105b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:30.365574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b57b42-4a5a-4b86-b127-76ae389105b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:30.366121 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b57b42-4a5a-4b86-b127-76ae389105b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:30.367253 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b57b42-4a5a-4b86-b127-76ae389105b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:30.368622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b57b42-4a5a-4b86-b127-76ae389105b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:30.381000 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b57b42-4a5a-4b86-b127-76ae389105b5 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:30.391428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d103fc67-5e3b-456d-8572-587aaa032282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '103fab04912b42a989f6fa75aa5e4e33'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:30.404725 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d103fc67-5e3b-456d-8572-587aaa032282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '103fab04912b42a989f6fa75aa5e4e33'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:30.444273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d103fc67-5e3b-456d-8572-587aaa032282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '103fab04912b42a989f6fa75aa5e4e33'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:30.446474 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 826/1647] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:30 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:30.447365 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b57b42-4a5a-4b86-b127-76ae389105b5 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:30.460747 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d103fc67-5e3b-456d-8572-587aaa032282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '103fab04912b42a989f6fa75aa5e4e33'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:30.462229 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d103fc67-5e3b-456d-8572-587aaa032282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '103fab04912b42a989f6fa75aa5e4e33'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:30.472356 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 822/1648] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:12:30 2022] DELETE /identity/v3/users/103fab04912b42a989f6fa75aa5e4e33 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:30.473401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103fc67-5e3b-456d-8572-587aaa032282 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:30.475538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d103fc67-5e3b-456d-8572-587aaa032282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:30.591525 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b57b42-4a5a-4b86-b127-76ae389105b5 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:30.668129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103fc67-5e3b-456d-8572-587aaa032282 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:30.683940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05b88d-4938-4428-a98b-034e5e546bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:30.686220 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05b88d-4938-4428-a98b-034e5e546bad admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:30.686608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05b88d-4938-4428-a98b-034e5e546bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:30.687018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05b88d-4938-4428-a98b-034e5e546bad admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:30.688432 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05b88d-4938-4428-a98b-034e5e546bad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:30.691934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05b88d-4938-4428-a98b-034e5e546bad admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:30.719277 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38b5a4-eade-4b3f-8a4f-6ef51da40e66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:30.721183 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38b5a4-eade-4b3f-8a4f-6ef51da40e66 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:30.721815 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38b5a4-eade-4b3f-8a4f-6ef51da40e66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:30.722288 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38b5a4-eade-4b3f-8a4f-6ef51da40e66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:30.725734 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-2c38b5a4-eade-4b3f-8a4f-6ef51da40e66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:30.742108 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 823/1649] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:30 2022] POST /identity/v3/users => generated 295 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:30.743745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e953b-0627-4131-9f9c-c69348bc7c38 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:30.746120 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e953b-0627-4131-9f9c-c69348bc7c38 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:30.746699 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e953b-0627-4131-9f9c-c69348bc7c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:30.785545 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-9c3e953b-0627-4131-9f9c-c69348bc7c38 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:30.786872 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9c3e953b-0627-4131-9f9c-c69348bc7c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:30.803377 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 824/1650] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:30.807337 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e953b-0627-4131-9f9c-c69348bc7c38 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:30.811802 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9c3e953b-0627-4131-9f9c-c69348bc7c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:30.841503 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38b5a4-eade-4b3f-8a4f-6ef51da40e66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:30.841979 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e953b-0627-4131-9f9c-c69348bc7c38 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:30.843154 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38b5a4-eade-4b3f-8a4f-6ef51da40e66 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:30.853938 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092e85a-bf01-4595-aec3-d31195449b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:30.855500 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092e85a-bf01-4595-aec3-d31195449b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:30.856515 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 827/1651] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:30 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:30.857536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092e85a-bf01-4595-aec3-d31195449b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:30.859489 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092e85a-bf01-4595-aec3-d31195449b99 admin admin] PATH_INFO: `/v3/users/9afe8e3912cf489a994e4f13e3914c8e` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:30.865924 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092e85a-bf01-4595-aec3-d31195449b99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9afe8e3912cf489a994e4f13e3914c8e)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:30.868518 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092e85a-bf01-4595-aec3-d31195449b99 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:30.868987 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38b5a4-eade-4b3f-8a4f-6ef51da40e66 service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:30.893658 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38b5a4-eade-4b3f-8a4f-6ef51da40e66 service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:30.906759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ae392-9ba0-4533-97df-7813f314a1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:30.907489 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ae392-9ba0-4533-97df-7813f314a1b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:30.907843 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ae392-9ba0-4533-97df-7813f314a1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:30.908186 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ae392-9ba0-4533-97df-7813f314a1b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:30.910016 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ae392-9ba0-4533-97df-7813f314a1b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:30.914308 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ae392-9ba0-4533-97df-7813f314a1b5 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:30.945924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 828/1652] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:30.947651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c557106-1fe6-486e-ac3c-62747ddfd1d6 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:30.948167 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c557106-1fe6-486e-ac3c-62747ddfd1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:30.948646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c557106-1fe6-486e-ac3c-62747ddfd1d6 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:30.950959 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 829/1653] 10.222.0.73 () {54 vars in 962 bytes} [Mon Aug 29 20:12:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:12:30.952367 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c557106-1fe6-486e-ac3c-62747ddfd1d6 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:30.954326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-7c557106-1fe6-486e-ac3c-62747ddfd1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:30.980726 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c092e85a-bf01-4595-aec3-d31195449b99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9afe8e3912cf489a994e4f13e3914c8e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:31.000026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c092e85a-bf01-4595-aec3-d31195449b99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9afe8e3912cf489a994e4f13e3914c8e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:31.007492 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:31.030634 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c557106-1fe6-486e-ac3c-62747ddfd1d6 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:31.037888 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcaf25-8367-449e-9d68-efe8397cbdee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:31.039616 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcaf25-8367-449e-9d68-efe8397cbdee service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:31.040056 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcaf25-8367-449e-9d68-efe8397cbdee service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:31.041043 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcaf25-8367-449e-9d68-efe8397cbdee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:31.042312 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c092e85a-bf01-4595-aec3-d31195449b99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9afe8e3912cf489a994e4f13e3914c8e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:31.044029 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:DELETE=19,keystone:SELECT=76,keystone:INSERT=5 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:31.068532 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c092e85a-bf01-4595-aec3-d31195449b99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9afe8e3912cf489a994e4f13e3914c8e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:31.069534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c092e85a-bf01-4595-aec3-d31195449b99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9afe8e3912cf489a994e4f13e3914c8e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:31.071022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fcaf25-8367-449e-9d68-efe8397cbdee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:31.075874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fcaf25-8367-449e-9d68-efe8397cbdee service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:31.088511 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 825/1654] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:12:30 2022] DELETE /identity/v3/users/9afe8e3912cf489a994e4f13e3914c8e => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:31.089933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092e85a-bf01-4595-aec3-d31195449b99 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:31.094057 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c092e85a-bf01-4595-aec3-d31195449b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:31.117908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 830/1655] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:31.120448 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0c6bb-131d-4ade-aeb5-e28c81ed9f39 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:31.120737 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0c6bb-131d-4ade-aeb5-e28c81ed9f39 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:31.121019 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0c6bb-131d-4ade-aeb5-e28c81ed9f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:31.176853 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-85e0c6bb-131d-4ade-aeb5-e28c81ed9f39 None None] MFA Rules not processed for user `95cf77a92a174ff8b0503baf5c7e63be`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:31.178186 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-85e0c6bb-131d-4ade-aeb5-e28c81ed9f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:31.278206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092e85a-bf01-4595-aec3-d31195449b99 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:31.286528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c780f9-4182-4468-9f2f-3ad61789a6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:31.287492 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c780f9-4182-4468-9f2f-3ad61789a6d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:31.287861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c780f9-4182-4468-9f2f-3ad61789a6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:31.288379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c780f9-4182-4468-9f2f-3ad61789a6d4 admin admin] PATH_INFO: `/v3/projects/dba3bce9a4e74b8e8f5be51b15df720d/users/cd489aa3dded4c149d10de2801798e27/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:31.308620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c780f9-4182-4468-9f2f-3ad61789a6d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dba3bce9a4e74b8e8f5be51b15df720d, user_id=cd489aa3dded4c149d10de2801798e27, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:31.312202 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c780f9-4182-4468-9f2f-3ad61789a6d4 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:31.337690 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 826/1656] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:31 2022] PUT /identity/v3/projects/dba3bce9a4e74b8e8f5be51b15df720d/users/cd489aa3dded4c149d10de2801798e27/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:31.340458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b99d2-f59d-4c70-809a-5f1f30d10c74 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:31.340908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b99d2-f59d-4c70-809a-5f1f30d10c74 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:31.341903 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b99d2-f59d-4c70-809a-5f1f30d10c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:31.357677 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 831/1657] 10.222.0.73 () {58 vars in 1059 bytes} [Mon Aug 29 20:12:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:31.358827 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0c6bb-131d-4ade-aeb5-e28c81ed9f39 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:31.375266 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0c6bb-131d-4ade-aeb5-e28c81ed9f39 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:31.383426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-cc3b99d2-f59d-4c70-809a-5f1f30d10c74 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:31.389421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-cc3b99d2-f59d-4c70-809a-5f1f30d10c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:31.498481 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ae0af-e68c-431a-92f4-61848b494176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:31.499717 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ae0af-e68c-431a-92f4-61848b494176 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:31.500256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ae0af-e68c-431a-92f4-61848b494176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:31.500742 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ae0af-e68c-431a-92f4-61848b494176 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:31.501660 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ae0af-e68c-431a-92f4-61848b494176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:31.503679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ae0af-e68c-431a-92f4-61848b494176 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:31.519678 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 832/1658] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:31.521341 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9509e-9ee4-4849-bc0c-79795700608e None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:31.521923 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9509e-9ee4-4849-bc0c-79795700608e None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:31.522528 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9509e-9ee4-4849-bc0c-79795700608e None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:31.524047 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 833/1659] 10.222.0.73 () {54 vars in 912 bytes} [Mon Aug 29 20:12:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:12:31.530408 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9509e-9ee4-4849-bc0c-79795700608e None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:31.532637 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 827/1660] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:31.536683 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cd10e-98f9-4863-9e7e-6da4ef85a778 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:31.536683 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cd10e-98f9-4863-9e7e-6da4ef85a778 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:31.536683 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cd10e-98f9-4863-9e7e-6da4ef85a778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:31.553304 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9509e-9ee4-4849-bc0c-79795700608e None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:31.559731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a8581-60d4-4cde-b35d-74fa866c5ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:31.560876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a8581-60d4-4cde-b35d-74fa866c5ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:31.562015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a8581-60d4-4cde-b35d-74fa866c5ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:31.562570 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a8581-60d4-4cde-b35d-74fa866c5ce7 admin admin] PATH_INFO: `/v3/projects/dba3bce9a4e74b8e8f5be51b15df720d/users/cd489aa3dded4c149d10de2801798e27/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:31.565388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a8581-60d4-4cde-b35d-74fa866c5ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dba3bce9a4e74b8e8f5be51b15df720d, user_id=cd489aa3dded4c149d10de2801798e27, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:31.576616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-0e5cd10e-98f9-4863-9e7e-6da4ef85a778 None None] MFA Rules not processed for user `95cf77a92a174ff8b0503baf5c7e63be`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:31.578182 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-0e5cd10e-98f9-4863-9e7e-6da4ef85a778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:31.590578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a8581-60d4-4cde-b35d-74fa866c5ce7 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:31.646613 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 834/1661] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:31 2022] PUT /identity/v3/projects/dba3bce9a4e74b8e8f5be51b15df720d/users/cd489aa3dded4c149d10de2801798e27/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:31.664740 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a8581-60d4-4cde-b35d-74fa866c5ce7 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:31.671689 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-8b5a8581-60d4-4cde-b35d-74fa866c5ce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:31.791369 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 828/1662] 10.222.0.73 () {58 vars in 1009 bytes} [Mon Aug 29 20:12:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:31.793414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475efc2-8153-4916-ac51-038634b6cd2d None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:31.794021 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475efc2-8153-4916-ac51-038634b6cd2d None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:31.795048 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475efc2-8153-4916-ac51-038634b6cd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:31.806523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:31.836095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a8581-60d4-4cde-b35d-74fa866c5ce7 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:31.837359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-c475efc2-8153-4916-ac51-038634b6cd2d None None] MFA Rules not processed for user `cd489aa3dded4c149d10de2801798e27`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:31.838859 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c475efc2-8153-4916-ac51-038634b6cd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:31.844843 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446d0b8-9c22-429b-8894-26c08b9bb023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:31.845493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446d0b8-9c22-429b-8894-26c08b9bb023 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:31.845761 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446d0b8-9c22-429b-8894-26c08b9bb023 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:31.846044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446d0b8-9c22-429b-8894-26c08b9bb023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:31.874488 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:31.971012 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f446d0b8-9c22-429b-8894-26c08b9bb023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:31.972277 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f446d0b8-9c22-429b-8894-26c08b9bb023 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:31.985137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 835/1663] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:31.986201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446d0b8-9c22-429b-8894-26c08b9bb023 service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:31.988202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f446d0b8-9c22-429b-8894-26c08b9bb023 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:32.000472 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 829/1664] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:32.049171 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475efc2-8153-4916-ac51-038634b6cd2d None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:32.051238 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c475efc2-8153-4916-ac51-038634b6cd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:32.120967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446d0b8-9c22-429b-8894-26c08b9bb023 service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:32.135525 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db753935-dfe7-408c-946b-6a8c6c542192 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:32.138053 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db753935-dfe7-408c-946b-6a8c6c542192 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:32.138320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db753935-dfe7-408c-946b-6a8c6c542192 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:32.140961 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db753935-dfe7-408c-946b-6a8c6c542192 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:32.241038 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475efc2-8153-4916-ac51-038634b6cd2d None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:32.252443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c5442-7a03-4e39-b826-fbbe30b93d4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:32.253085 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c5442-7a03-4e39-b826-fbbe30b93d4a service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:32.256727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c5442-7a03-4e39-b826-fbbe30b93d4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:32.257397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c5442-7a03-4e39-b826-fbbe30b93d4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:32.284798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c5442-7a03-4e39-b826-fbbe30b93d4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:32.289441 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c5442-7a03-4e39-b826-fbbe30b93d4a service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:32.332762 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 830/1665] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:32.357803 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db753935-dfe7-408c-946b-6a8c6c542192 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:32.359488 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db753935-dfe7-408c-946b-6a8c6c542192 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:32.376000 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 836/1666] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:32.480759 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c5442-7a03-4e39-b826-fbbe30b93d4a service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:32.492117 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c5442-7a03-4e39-b826-fbbe30b93d4a service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:32.497335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46414e91-c0e2-4c19-950e-b9b87620636c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:32.498100 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46414e91-c0e2-4c19-950e-b9b87620636c service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:32.498378 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46414e91-c0e2-4c19-950e-b9b87620636c service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:32.498650 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46414e91-c0e2-4c19-950e-b9b87620636c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:32.501310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-46414e91-c0e2-4c19-950e-b9b87620636c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:32.607517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5390c53-85ac-4eba-8239-1fa23688050f None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:32.608010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5390c53-85ac-4eba-8239-1fa23688050f None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:32.608471 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5390c53-85ac-4eba-8239-1fa23688050f None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:32.610200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 837/1667] 10.222.0.73 () {54 vars in 912 bytes} [Mon Aug 29 20:12:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:12:32.623803 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e032dd4-ac74-4856-aee3-939d46dde032 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:32.624450 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e032dd4-ac74-4856-aee3-939d46dde032 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:32.624973 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e032dd4-ac74-4856-aee3-939d46dde032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:32.663702 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-8e032dd4-ac74-4856-aee3-939d46dde032 None None] MFA Rules not processed for user `95cf77a92a174ff8b0503baf5c7e63be`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:32.665236 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-8e032dd4-ac74-4856-aee3-939d46dde032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:32.692651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46414e91-c0e2-4c19-950e-b9b87620636c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:32.694536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46414e91-c0e2-4c19-950e-b9b87620636c service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:32.700099 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 838/1668] 10.222.0.73 () {58 vars in 1009 bytes} [Mon Aug 29 20:12:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:32.731669 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 831/1669] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:32.786747 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e032dd4-ac74-4856-aee3-939d46dde032 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:32.799752 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46414e91-c0e2-4c19-950e-b9b87620636c service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:32.801788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-46414e91-c0e2-4c19-950e-b9b87620636c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:32.827759 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46414e91-c0e2-4c19-950e-b9b87620636c service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:32.828144 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e032dd4-ac74-4856-aee3-939d46dde032 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:32.838908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f6aa6-42f0-45fc-8b34-b986a029417a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:32.839849 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f6aa6-42f0-45fc-8b34-b986a029417a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:32.840285 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f6aa6-42f0-45fc-8b34-b986a029417a admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:32.842819 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f6aa6-42f0-45fc-8b34-b986a029417a admin admin] PATH_INFO: `/v3/projects/0ef97e4835ab48d8935356ba4afa5918` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:32.844361 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f6aa6-42f0-45fc-8b34-b986a029417a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ef97e4835ab48d8935356ba4afa5918)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:32.846360 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: WARNING py.warnings [None req-287f6aa6-42f0-45fc-8b34-b986a029417a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:12:32.846360 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: warnings.warn(msg) Aug 29 20:12:32.846360 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: Aug 29 20:12:32.846705 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc4128-e08a-4c04-af82-c11cf24d5065 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:32.848200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc4128-e08a-4c04-af82-c11cf24d5065 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:32.848743 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc4128-e08a-4c04-af82-c11cf24d5065 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:32.849014 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f6aa6-42f0-45fc-8b34-b986a029417a admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:32.849150 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc4128-e08a-4c04-af82-c11cf24d5065 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:32.878586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc4128-e08a-4c04-af82-c11cf24d5065 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:32.885434 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc4128-e08a-4c04-af82-c11cf24d5065 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:32.919470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 839/1670] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:32.955535 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:33.060141 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-287f6aa6-42f0-45fc-8b34-b986a029417a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ef97e4835ab48d8935356ba4afa5918', 'request_id': 'req-287f6aa6-42f0-45fc-8b34-b986a029417a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:33.133198 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-287f6aa6-42f0-45fc-8b34-b986a029417a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ef97e4835ab48d8935356ba4afa5918', 'request_id': 'req-287f6aa6-42f0-45fc-8b34-b986a029417a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:33.155747 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-287f6aa6-42f0-45fc-8b34-b986a029417a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ef97e4835ab48d8935356ba4afa5918', 'request_id': 'req-287f6aa6-42f0-45fc-8b34-b986a029417a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:33.178487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-287f6aa6-42f0-45fc-8b34-b986a029417a admin admin] The token cache is being invalidate because project 0ef97e4835ab48d8935356ba4afa5918 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:12:33.178786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-287f6aa6-42f0-45fc-8b34-b986a029417a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:33.180503 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-287f6aa6-42f0-45fc-8b34-b986a029417a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:33.190063 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 832/1671] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:12:32 2022] DELETE /identity/v3/projects/0ef97e4835ab48d8935356ba4afa5918 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:33.194073 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc4128-e08a-4c04-af82-c11cf24d5065 service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:33.217030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc4128-e08a-4c04-af82-c11cf24d5065 service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:33.276200 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f6aa6-42f0-45fc-8b34-b986a029417a admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:33.277270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-287f6aa6-42f0-45fc-8b34-b986a029417a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:33.291666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:33.431302 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be0d14-5cd6-4d60-a98b-91766e1eed9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:33.434257 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be0d14-5cd6-4d60-a98b-91766e1eed9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:33.434257 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be0d14-5cd6-4d60-a98b-91766e1eed9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:33.434682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be0d14-5cd6-4d60-a98b-91766e1eed9b admin admin] PATH_INFO: `/v3/users/3280ccc382934a77b1f3820edcc11830` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:33.443063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69be0d14-5cd6-4d60-a98b-91766e1eed9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3280ccc382934a77b1f3820edcc11830)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:33.449654 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: WARNING py.warnings [None req-69be0d14-5cd6-4d60-a98b-91766e1eed9b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:12:33.449654 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: warnings.warn(msg) Aug 29 20:12:33.449654 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: Aug 29 20:12:33.452743 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69be0d14-5cd6-4d60-a98b-91766e1eed9b admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:33.507901 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f6aa6-42f0-45fc-8b34-b986a029417a admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:33.524793 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942c4d3-74b5-45da-9699-e826a9c4db7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:33.525572 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942c4d3-74b5-45da-9699-e826a9c4db7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:33.527917 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942c4d3-74b5-45da-9699-e826a9c4db7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:33.528325 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942c4d3-74b5-45da-9699-e826a9c4db7e admin admin] PATH_INFO: `/v3/users/8a55334adb294937abaf0c2bef36be3e` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:33.533460 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942c4d3-74b5-45da-9699-e826a9c4db7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a55334adb294937abaf0c2bef36be3e)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:33.537542 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942c4d3-74b5-45da-9699-e826a9c4db7e admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:33.696799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-69be0d14-5cd6-4d60-a98b-91766e1eed9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3280ccc382934a77b1f3820edcc11830'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:33.728908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9942c4d3-74b5-45da-9699-e826a9c4db7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a55334adb294937abaf0c2bef36be3e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:33.749129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9942c4d3-74b5-45da-9699-e826a9c4db7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a55334adb294937abaf0c2bef36be3e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:33.765375 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-69be0d14-5cd6-4d60-a98b-91766e1eed9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3280ccc382934a77b1f3820edcc11830'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:33.782330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-69be0d14-5cd6-4d60-a98b-91766e1eed9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3280ccc382934a77b1f3820edcc11830'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:33.789072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9942c4d3-74b5-45da-9699-e826a9c4db7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a55334adb294937abaf0c2bef36be3e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:33.835081 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9942c4d3-74b5-45da-9699-e826a9c4db7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a55334adb294937abaf0c2bef36be3e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:33.836044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-69be0d14-5cd6-4d60-a98b-91766e1eed9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3280ccc382934a77b1f3820edcc11830'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:33.836802 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9942c4d3-74b5-45da-9699-e826a9c4db7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a55334adb294937abaf0c2bef36be3e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:33.837414 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-69be0d14-5cd6-4d60-a98b-91766e1eed9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3280ccc382934a77b1f3820edcc11830'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:33.872837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 833/1672] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:12:33 2022] DELETE /identity/v3/users/8a55334adb294937abaf0c2bef36be3e => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:12:33.883003 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 840/1673] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:12:33 2022] DELETE /identity/v3/users/3280ccc382934a77b1f3820edcc11830 => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:33.883643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942c4d3-74b5-45da-9699-e826a9c4db7e admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:33.886448 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4c3d4-c5dc-4c74-a43f-8e251d2d7a97 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:33.886733 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4c3d4-c5dc-4c74-a43f-8e251d2d7a97 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:33.890496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9942c4d3-74b5-45da-9699-e826a9c4db7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:33.891450 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4c3d4-c5dc-4c74-a43f-8e251d2d7a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:33.996442 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-2bd4c3d4-c5dc-4c74-a43f-8e251d2d7a97 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:34.004387 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-2bd4c3d4-c5dc-4c74-a43f-8e251d2d7a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:34.106445 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:34.224414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942c4d3-74b5-45da-9699-e826a9c4db7e admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:34.226071 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 841/1674] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:34.237455 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb890cd-d370-4d0d-8412-3a8df8f2fabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:34.241365 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb890cd-d370-4d0d-8412-3a8df8f2fabc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:34.242715 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb890cd-d370-4d0d-8412-3a8df8f2fabc admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:34.243301 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb890cd-d370-4d0d-8412-3a8df8f2fabc admin admin] PATH_INFO: `/v3/projects/e02f019a5ffa4653abf9f7633e4251fb` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:34.245324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb890cd-d370-4d0d-8412-3a8df8f2fabc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e02f019a5ffa4653abf9f7633e4251fb)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:34.249249 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb890cd-d370-4d0d-8412-3a8df8f2fabc admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:34.284080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4c3d4-c5dc-4c74-a43f-8e251d2d7a97 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:34.286311 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-2bd4c3d4-c5dc-4c74-a43f-8e251d2d7a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:34.397005 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-cfb890cd-d370-4d0d-8412-3a8df8f2fabc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e02f019a5ffa4653abf9f7633e4251fb', 'request_id': 'req-cfb890cd-d370-4d0d-8412-3a8df8f2fabc'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:34.425347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-cfb890cd-d370-4d0d-8412-3a8df8f2fabc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e02f019a5ffa4653abf9f7633e4251fb', 'request_id': 'req-cfb890cd-d370-4d0d-8412-3a8df8f2fabc'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:34.446494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-cfb890cd-d370-4d0d-8412-3a8df8f2fabc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e02f019a5ffa4653abf9f7633e4251fb', 'request_id': 'req-cfb890cd-d370-4d0d-8412-3a8df8f2fabc'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:34.491818 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-cfb890cd-d370-4d0d-8412-3a8df8f2fabc admin admin] The token cache is being invalidate because project e02f019a5ffa4653abf9f7633e4251fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:12:34.492433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-cfb890cd-d370-4d0d-8412-3a8df8f2fabc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:34.493531 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-cfb890cd-d370-4d0d-8412-3a8df8f2fabc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:34.495978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 834/1675] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:12:33 2022] DELETE /identity/v3/projects/e02f019a5ffa4653abf9f7633e4251fb => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:34.500238 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb890cd-d370-4d0d-8412-3a8df8f2fabc admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:34.504305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-cfb890cd-d370-4d0d-8412-3a8df8f2fabc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:34.521169 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:34.538512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4c3d4-c5dc-4c74-a43f-8e251d2d7a97 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:34.720988 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb890cd-d370-4d0d-8412-3a8df8f2fabc admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:34.727006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e657a58-e2fb-4a8a-a910-cfac919ebb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:34.727993 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e657a58-e2fb-4a8a-a910-cfac919ebb51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:34.728739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e657a58-e2fb-4a8a-a910-cfac919ebb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:34.728950 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e657a58-e2fb-4a8a-a910-cfac919ebb51 admin admin] PATH_INFO: `/v3/users/d8d0bc6d996f44279fd6bc879a88e52d` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:34.730560 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e657a58-e2fb-4a8a-a910-cfac919ebb51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8d0bc6d996f44279fd6bc879a88e52d)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:34.741284 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e657a58-e2fb-4a8a-a910-cfac919ebb51 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:34.772017 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a653dc-dcad-496e-ac7a-7fef06292817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:34.772910 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a653dc-dcad-496e-ac7a-7fef06292817 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:34.773207 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a653dc-dcad-496e-ac7a-7fef06292817 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:34.780639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a653dc-dcad-496e-ac7a-7fef06292817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:34.789053 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-28a653dc-dcad-496e-ac7a-7fef06292817 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:34.855307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a653dc-dcad-496e-ac7a-7fef06292817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:34.859711 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a653dc-dcad-496e-ac7a-7fef06292817 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:34.894855 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 842/1676] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:34.910887 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-2e657a58-e2fb-4a8a-a910-cfac919ebb51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8d0bc6d996f44279fd6bc879a88e52d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:34.925404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-2e657a58-e2fb-4a8a-a910-cfac919ebb51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8d0bc6d996f44279fd6bc879a88e52d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:34.945172 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:34.956713 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-2e657a58-e2fb-4a8a-a910-cfac919ebb51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8d0bc6d996f44279fd6bc879a88e52d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:34.974067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-2e657a58-e2fb-4a8a-a910-cfac919ebb51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8d0bc6d996f44279fd6bc879a88e52d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:34.975848 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-2e657a58-e2fb-4a8a-a910-cfac919ebb51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8d0bc6d996f44279fd6bc879a88e52d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:35.000967 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 835/1677] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:12:34 2022] DELETE /identity/v3/users/d8d0bc6d996f44279fd6bc879a88e52d => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:35.134005 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a653dc-dcad-496e-ac7a-7fef06292817 service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:35.140442 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-28a653dc-dcad-496e-ac7a-7fef06292817 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:35.165191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a653dc-dcad-496e-ac7a-7fef06292817 service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:35.351836 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386cdc5-a056-40bc-821d-1a5559a3a706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:35.353386 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386cdc5-a056-40bc-821d-1a5559a3a706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:35.354294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386cdc5-a056-40bc-821d-1a5559a3a706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:35.354947 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386cdc5-a056-40bc-821d-1a5559a3a706 admin admin] PATH_INFO: `/v3/users/897041db45a84d3fbb1b680c83efafdf` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:35.359108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386cdc5-a056-40bc-821d-1a5559a3a706 admin admin] RBAC: Authorizing `identity:delete_user(user_id=897041db45a84d3fbb1b680c83efafdf)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:35.370728 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386cdc5-a056-40bc-821d-1a5559a3a706 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:35.434801 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e657a58-e2fb-4a8a-a910-cfac919ebb51 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:35.438350 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-2e657a58-e2fb-4a8a-a910-cfac919ebb51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:35.469158 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e657a58-e2fb-4a8a-a910-cfac919ebb51 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:35.487263 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff902331-cd22-4cd0-9504-ff0180f08edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:35.488273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff902331-cd22-4cd0-9504-ff0180f08edb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:35.488849 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff902331-cd22-4cd0-9504-ff0180f08edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:35.489341 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff902331-cd22-4cd0-9504-ff0180f08edb admin admin] PATH_INFO: `/v3/projects/3e52facf2a124b95928f1145a70ea2ae` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:35.491427 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff902331-cd22-4cd0-9504-ff0180f08edb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e52facf2a124b95928f1145a70ea2ae)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:35.494563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff902331-cd22-4cd0-9504-ff0180f08edb admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:35.517500 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1386cdc5-a056-40bc-821d-1a5559a3a706 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '897041db45a84d3fbb1b680c83efafdf'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:35.561979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1386cdc5-a056-40bc-821d-1a5559a3a706 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '897041db45a84d3fbb1b680c83efafdf'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:35.584743 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1386cdc5-a056-40bc-821d-1a5559a3a706 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '897041db45a84d3fbb1b680c83efafdf'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:35.604239 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1386cdc5-a056-40bc-821d-1a5559a3a706 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '897041db45a84d3fbb1b680c83efafdf'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:35.605760 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1386cdc5-a056-40bc-821d-1a5559a3a706 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '897041db45a84d3fbb1b680c83efafdf'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:35.621733 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 843/1678] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:12:35 2022] DELETE /identity/v3/users/897041db45a84d3fbb1b680c83efafdf => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:35.626184 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-ff902331-cd22-4cd0-9504-ff0180f08edb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e52facf2a124b95928f1145a70ea2ae', 'request_id': 'req-ff902331-cd22-4cd0-9504-ff0180f08edb'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:35.626996 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c4c17-5e78-4f42-98d0-75b996404437 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:35.627295 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c4c17-5e78-4f42-98d0-75b996404437 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:35.627574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c4c17-5e78-4f42-98d0-75b996404437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:35.644413 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-ff902331-cd22-4cd0-9504-ff0180f08edb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e52facf2a124b95928f1145a70ea2ae', 'request_id': 'req-ff902331-cd22-4cd0-9504-ff0180f08edb'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:35.662793 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-ff902331-cd22-4cd0-9504-ff0180f08edb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e52facf2a124b95928f1145a70ea2ae', 'request_id': 'req-ff902331-cd22-4cd0-9504-ff0180f08edb'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:35.678563 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-c03c4c17-5e78-4f42-98d0-75b996404437 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:35.682476 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c03c4c17-5e78-4f42-98d0-75b996404437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:35.691365 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-ff902331-cd22-4cd0-9504-ff0180f08edb admin admin] The token cache is being invalidate because project 3e52facf2a124b95928f1145a70ea2ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:12:35.691672 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-ff902331-cd22-4cd0-9504-ff0180f08edb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:35.694749 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-ff902331-cd22-4cd0-9504-ff0180f08edb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:35.703945 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 836/1679] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:12:35 2022] DELETE /identity/v3/projects/3e52facf2a124b95928f1145a70ea2ae => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:35.706400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69323ec5-f7bd-465b-a1c3-4bca53cd79a9 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:35.710865 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69323ec5-f7bd-465b-a1c3-4bca53cd79a9 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:35.711337 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69323ec5-f7bd-465b-a1c3-4bca53cd79a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:35.757456 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-69323ec5-f7bd-465b-a1c3-4bca53cd79a9 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:35.762197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-69323ec5-f7bd-465b-a1c3-4bca53cd79a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:35.788966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:35.870037 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 844/1680] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:12:36.015472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c4c17-5e78-4f42-98d0-75b996404437 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:36.017894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c03c4c17-5e78-4f42-98d0-75b996404437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:36.029253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 837/1681] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:36.046024 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc45a9-4425-48ee-8401-6899dbc3a081 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:36.046613 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc45a9-4425-48ee-8401-6899dbc3a081 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:36.047635 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc45a9-4425-48ee-8401-6899dbc3a081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:36.110571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-81fc45a9-4425-48ee-8401-6899dbc3a081 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:36.112576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-81fc45a9-4425-48ee-8401-6899dbc3a081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:36.151048 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 838/1682] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:12:36.160077 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc45a9-4425-48ee-8401-6899dbc3a081 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:36.213360 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc45a9-4425-48ee-8401-6899dbc3a081 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:36.233417 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2bcf7-a644-42f1-a743-fc3069b19932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:36.234126 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2bcf7-a644-42f1-a743-fc3069b19932 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:36.234411 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2bcf7-a644-42f1-a743-fc3069b19932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:36.234681 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2bcf7-a644-42f1-a743-fc3069b19932 admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:36.235347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d2bcf7-a644-42f1-a743-fc3069b19932 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:12:36.235652 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d2bcf7-a644-42f1-a743-fc3069b19932 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:36.256092 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d2bcf7-a644-42f1-a743-fc3069b19932 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:36.260542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c4c17-5e78-4f42-98d0-75b996404437 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:36.270536 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662221b-fdd5-4a19-ab52-6fe6ce493e5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:36.272139 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662221b-fdd5-4a19-ab52-6fe6ce493e5a service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:36.273639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662221b-fdd5-4a19-ab52-6fe6ce493e5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:36.274101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662221b-fdd5-4a19-ab52-6fe6ce493e5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:36.294965 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 839/1683] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:12:36.300893 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2bcf7-a644-42f1-a743-fc3069b19932 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:36.304963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3662221b-fdd5-4a19-ab52-6fe6ce493e5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:36.308405 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3662221b-fdd5-4a19-ab52-6fe6ce493e5a service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:36.327860 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2bcf7-a644-42f1-a743-fc3069b19932 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:36.339124 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 845/1684] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:36.340630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d773cc-def2-42ef-a5b5-18bd00e7c471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:36.342688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d773cc-def2-42ef-a5b5-18bd00e7c471 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:36.343052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d773cc-def2-42ef-a5b5-18bd00e7c471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:36.343332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d773cc-def2-42ef-a5b5-18bd00e7c471 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:36.344373 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d773cc-def2-42ef-a5b5-18bd00e7c471 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:36.346137 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d773cc-def2-42ef-a5b5-18bd00e7c471 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:36.391475 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 840/1685] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:36 2022] POST /identity/v3/projects => generated 363 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:36.397136 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662221b-fdd5-4a19-ab52-6fe6ce493e5a service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:36.410830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662221b-fdd5-4a19-ab52-6fe6ce493e5a service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:36.499947 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d773cc-def2-42ef-a5b5-18bd00e7c471 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:36.502704 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-e6d773cc-def2-42ef-a5b5-18bd00e7c471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:36.574984 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:36.619946 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ac776-b310-49e0-a7c2-8cc5dfa7a835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:36.620648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ac776-b310-49e0-a7c2-8cc5dfa7a835 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:36.620934 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ac776-b310-49e0-a7c2-8cc5dfa7a835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:36.621364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ac776-b310-49e0-a7c2-8cc5dfa7a835 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:36.622045 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ac776-b310-49e0-a7c2-8cc5dfa7a835 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:36.630557 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ac776-b310-49e0-a7c2-8cc5dfa7a835 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:36.673502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d773cc-def2-42ef-a5b5-18bd00e7c471 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:36.681082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 846/1686] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:36 2022] POST /identity/v3/users => generated 282 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:36.682428 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ac776-b310-49e0-a7c2-8cc5dfa7a835 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:36.684326 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8874649-70c4-4fec-8508-8d775cc09fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:36.685110 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8874649-70c4-4fec-8508-8d775cc09fd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:36.685393 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8874649-70c4-4fec-8508-8d775cc09fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:36.685724 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8874649-70c4-4fec-8508-8d775cc09fd4 admin admin] PATH_INFO: `/v3/projects/cbb0922aa97f4f39bb7a426a0a94ffb1` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:36.690407 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8874649-70c4-4fec-8508-8d775cc09fd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbb0922aa97f4f39bb7a426a0a94ffb1)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:36.694544 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8874649-70c4-4fec-8508-8d775cc09fd4 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:36.710272 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ac776-b310-49e0-a7c2-8cc5dfa7a835 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:36.724992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4a5fb-4478-4057-a97e-9e974fe68b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:36.726040 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4a5fb-4478-4057-a97e-9e974fe68b4d admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:36.726546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4a5fb-4478-4057-a97e-9e974fe68b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:36.727015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4a5fb-4478-4057-a97e-9e974fe68b4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:36.727889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa4a5fb-4478-4057-a97e-9e974fe68b4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:36.730557 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa4a5fb-4478-4057-a97e-9e974fe68b4d admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:36.759205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 847/1687] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:36.762103 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4a5fb-4478-4057-a97e-9e974fe68b4d admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:36.789228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4a5fb-4478-4057-a97e-9e974fe68b4d admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:36.791600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c8874649-70c4-4fec-8508-8d775cc09fd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbb0922aa97f4f39bb7a426a0a94ffb1', 'request_id': 'req-c8874649-70c4-4fec-8508-8d775cc09fd4'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:36.803421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c8874649-70c4-4fec-8508-8d775cc09fd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbb0922aa97f4f39bb7a426a0a94ffb1', 'request_id': 'req-c8874649-70c4-4fec-8508-8d775cc09fd4'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:36.821974 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c8874649-70c4-4fec-8508-8d775cc09fd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbb0922aa97f4f39bb7a426a0a94ffb1', 'request_id': 'req-c8874649-70c4-4fec-8508-8d775cc09fd4'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:36.843346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c8874649-70c4-4fec-8508-8d775cc09fd4 admin admin] The token cache is being invalidate because project cbb0922aa97f4f39bb7a426a0a94ffb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:12:36.843690 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c8874649-70c4-4fec-8508-8d775cc09fd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:36.844352 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c8874649-70c4-4fec-8508-8d775cc09fd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:36.847170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 841/1688] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:12:36 2022] DELETE /identity/v3/projects/cbb0922aa97f4f39bb7a426a0a94ffb1 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:36.854724 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d900a-3667-45e3-b975-74cd0db58a94 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:36.854724 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d900a-3667-45e3-b975-74cd0db58a94 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:36.854974 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d900a-3667-45e3-b975-74cd0db58a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:36.900417 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:36.908127 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-4c6d900a-3667-45e3-b975-74cd0db58a94 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:36.909384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-4c6d900a-3667-45e3-b975-74cd0db58a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:36.936577 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1421c1-00ab-4afb-a3cd-560f7ea3dd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:36.938082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1421c1-00ab-4afb-a3cd-560f7ea3dd76 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:36.938852 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1421c1-00ab-4afb-a3cd-560f7ea3dd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:36.939328 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1421c1-00ab-4afb-a3cd-560f7ea3dd76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:36.940350 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1421c1-00ab-4afb-a3cd-560f7ea3dd76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:36.945401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1421c1-00ab-4afb-a3cd-560f7ea3dd76 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:36.976384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 848/1689] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:36.977600 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1421c1-00ab-4afb-a3cd-560f7ea3dd76 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:36.991476 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-8d1421c1-00ab-4afb-a3cd-560f7ea3dd76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:37.002906 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 842/1690] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:37.005188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bceefa-3de6-425a-ac60-c7884c98f39e None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:37.005482 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bceefa-3de6-425a-ac60-c7884c98f39e None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:37.005825 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bceefa-3de6-425a-ac60-c7884c98f39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:37.032356 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1421c1-00ab-4afb-a3cd-560f7ea3dd76 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:37.037335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-11bceefa-3de6-425a-ac60-c7884c98f39e None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:37.038739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-11bceefa-3de6-425a-ac60-c7884c98f39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:37.039526 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a263a-1c8b-4a0d-b509-51e5aa4ef8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:37.040598 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a263a-1c8b-4a0d-b509-51e5aa4ef8c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:37.041053 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a263a-1c8b-4a0d-b509-51e5aa4ef8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:37.041960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a263a-1c8b-4a0d-b509-51e5aa4ef8c5 admin admin] PATH_INFO: `/v3/projects/445d138b2f984ffebd1dd3aef0b97ff5/users/3056b78f7d7f4c3e9bcba1a420990e44/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:37.051969 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 843/1691] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:37.057356 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bceefa-3de6-425a-ac60-c7884c98f39e None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:37.079963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bceefa-3de6-425a-ac60-c7884c98f39e None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:37.080880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a263a-1c8b-4a0d-b509-51e5aa4ef8c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=445d138b2f984ffebd1dd3aef0b97ff5, user_id=3056b78f7d7f4c3e9bcba1a420990e44, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:37.090907 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a263a-1c8b-4a0d-b509-51e5aa4ef8c5 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:37.099725 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57793e01-6296-4234-8c2a-86ebafca56c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:37.100357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57793e01-6296-4234-8c2a-86ebafca56c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:37.100642 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57793e01-6296-4234-8c2a-86ebafca56c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:37.101045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57793e01-6296-4234-8c2a-86ebafca56c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:37.101570 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57793e01-6296-4234-8c2a-86ebafca56c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:12:37.103870 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57793e01-6296-4234-8c2a-86ebafca56c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:37.107027 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57793e01-6296-4234-8c2a-86ebafca56c9 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:37.128642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 849/1692] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:36 2022] PUT /identity/v3/projects/445d138b2f984ffebd1dd3aef0b97ff5/users/3056b78f7d7f4c3e9bcba1a420990e44/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:37.131550 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78d890-ea9f-4fc2-a1df-d159cc1695ef None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:37.131841 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78d890-ea9f-4fc2-a1df-d159cc1695ef None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:37.132123 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78d890-ea9f-4fc2-a1df-d159cc1695ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:37.148957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 844/1693] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:12:37.150665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57793e01-6296-4234-8c2a-86ebafca56c9 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:37.184647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57793e01-6296-4234-8c2a-86ebafca56c9 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:37.208988 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-3a78d890-ea9f-4fc2-a1df-d159cc1695ef None None] MFA Rules not processed for user `3056b78f7d7f4c3e9bcba1a420990e44`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:37.209962 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-3a78d890-ea9f-4fc2-a1df-d159cc1695ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:37.300548 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:37.359347 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 850/1694] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:12:37.362709 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78d890-ea9f-4fc2-a1df-d159cc1695ef None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:37.375971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4ae88-d12d-4c63-bc23-484d51d1c8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:37.376618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4ae88-d12d-4c63-bc23-484d51d1c8c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:37.376902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4ae88-d12d-4c63-bc23-484d51d1c8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:37.377618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4ae88-d12d-4c63-bc23-484d51d1c8c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:37.378294 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4ae88-d12d-4c63-bc23-484d51d1c8c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:37.384167 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4ae88-d12d-4c63-bc23-484d51d1c8c0 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:37.471721 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 845/1695] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:37 2022] POST /identity/v3/projects => generated 371 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:37.475695 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4ae88-d12d-4c63-bc23-484d51d1c8c0 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:37.538930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78d890-ea9f-4fc2-a1df-d159cc1695ef None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:37.643070 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4ae88-d12d-4c63-bc23-484d51d1c8c0 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:37.661735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a1161-8219-45a2-8ddd-0f7a56d17b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:37.662299 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a1161-8219-45a2-8ddd-0f7a56d17b6a admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:37.662580 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a1161-8219-45a2-8ddd-0f7a56d17b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:37.662859 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a1161-8219-45a2-8ddd-0f7a56d17b6a admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:37.663507 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a1161-8219-45a2-8ddd-0f7a56d17b6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:37.666344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a1161-8219-45a2-8ddd-0f7a56d17b6a admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:37.681389 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:37.698161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53555c89-4cb0-4c2f-9b02-fcf3c704a829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:37.699870 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53555c89-4cb0-4c2f-9b02-fcf3c704a829 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:37.701620 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53555c89-4cb0-4c2f-9b02-fcf3c704a829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:37.702540 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53555c89-4cb0-4c2f-9b02-fcf3c704a829 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:37.703647 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53555c89-4cb0-4c2f-9b02-fcf3c704a829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:37.709745 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53555c89-4cb0-4c2f-9b02-fcf3c704a829 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:37.715899 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 846/1696] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:37 2022] POST /identity/v3/users => generated 286 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:37.732480 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a1161-8219-45a2-8ddd-0f7a56d17b6a admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:37.770697 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a1161-8219-45a2-8ddd-0f7a56d17b6a admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:37.773797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 851/1697] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:37 2022] POST /identity/v3/projects => generated 365 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:37.776790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53555c89-4cb0-4c2f-9b02-fcf3c704a829 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:37.782663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-53555c89-4cb0-4c2f-9b02-fcf3c704a829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:37.934738 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53555c89-4cb0-4c2f-9b02-fcf3c704a829 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:37.946350 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615b44b-d428-4de9-927b-02406ddb8a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:37.947010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615b44b-d428-4de9-927b-02406ddb8a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:37.947286 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615b44b-d428-4de9-927b-02406ddb8a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:37.947587 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615b44b-d428-4de9-927b-02406ddb8a8c admin admin] PATH_INFO: `/v3/projects/5926f39634e84952a23ff4375cbfc22f` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:37.949686 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615b44b-d428-4de9-927b-02406ddb8a8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5926f39634e84952a23ff4375cbfc22f)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:37.952796 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615b44b-d428-4de9-927b-02406ddb8a8c admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:37.995896 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cbddff-d9c9-4026-8b3b-dba8f2b1a8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:37.997837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbddff-d9c9-4026-8b3b-dba8f2b1a8fc admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:37.998462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbddff-d9c9-4026-8b3b-dba8f2b1a8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:38.000906 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbddff-d9c9-4026-8b3b-dba8f2b1a8fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:38.002913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cbddff-d9c9-4026-8b3b-dba8f2b1a8fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:38.004683 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cbddff-d9c9-4026-8b3b-dba8f2b1a8fc admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:38.045938 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 847/1698] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:38.050459 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cbddff-d9c9-4026-8b3b-dba8f2b1a8fc admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:38.063698 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-3615b44b-d428-4de9-927b-02406ddb8a8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5926f39634e84952a23ff4375cbfc22f', 'request_id': 'req-3615b44b-d428-4de9-927b-02406ddb8a8c'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:38.079179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-3615b44b-d428-4de9-927b-02406ddb8a8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5926f39634e84952a23ff4375cbfc22f', 'request_id': 'req-3615b44b-d428-4de9-927b-02406ddb8a8c'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:38.085479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cbddff-d9c9-4026-8b3b-dba8f2b1a8fc admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:38.103610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-3615b44b-d428-4de9-927b-02406ddb8a8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5926f39634e84952a23ff4375cbfc22f', 'request_id': 'req-3615b44b-d428-4de9-927b-02406ddb8a8c'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:38.127141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-3615b44b-d428-4de9-927b-02406ddb8a8c admin admin] The token cache is being invalidate because project 5926f39634e84952a23ff4375cbfc22f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:12:38.127297 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-3615b44b-d428-4de9-927b-02406ddb8a8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:38.128773 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-3615b44b-d428-4de9-927b-02406ddb8a8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:38.132080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 852/1699] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:12:37 2022] DELETE /identity/v3/projects/5926f39634e84952a23ff4375cbfc22f => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:38.133303 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615b44b-d428-4de9-927b-02406ddb8a8c admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:38.135803 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-3615b44b-d428-4de9-927b-02406ddb8a8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:38.191498 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:38.220940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182a031-06d6-4e7f-9322-77ebc608d93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:38.221624 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182a031-06d6-4e7f-9322-77ebc608d93c admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:38.222199 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182a031-06d6-4e7f-9322-77ebc608d93c admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:38.222930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182a031-06d6-4e7f-9322-77ebc608d93c admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:38.223202 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6182a031-06d6-4e7f-9322-77ebc608d93c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:38.229540 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6182a031-06d6-4e7f-9322-77ebc608d93c admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:38.279733 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 848/1700] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:38 2022] POST /identity/v3/users => generated 289 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:38.284728 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ee8b6-3519-4dd4-bb8b-052cc5c32b43 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:38.285988 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ee8b6-3519-4dd4-bb8b-052cc5c32b43 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:38.286338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ee8b6-3519-4dd4-bb8b-052cc5c32b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:38.304796 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615b44b-d428-4de9-927b-02406ddb8a8c admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:38.323130 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eeaa65-f001-48ce-b023-6d8ffd33fce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:38.324563 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eeaa65-f001-48ce-b023-6d8ffd33fce7 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:38.324885 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eeaa65-f001-48ce-b023-6d8ffd33fce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:38.325275 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eeaa65-f001-48ce-b023-6d8ffd33fce7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:38.326516 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eeaa65-f001-48ce-b023-6d8ffd33fce7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:38.330788 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eeaa65-f001-48ce-b023-6d8ffd33fce7 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:38.343528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-728ee8b6-3519-4dd4-bb8b-052cc5c32b43 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:38.345505 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-728ee8b6-3519-4dd4-bb8b-052cc5c32b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:38.378966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 853/1701] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:38.381749 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eeaa65-f001-48ce-b023-6d8ffd33fce7 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:38.395589 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 849/1702] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:38.398154 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ee8b6-3519-4dd4-bb8b-052cc5c32b43 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:38.399041 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f9eeaa65-f001-48ce-b023-6d8ffd33fce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:38.402252 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-728ee8b6-3519-4dd4-bb8b-052cc5c32b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:38.432197 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eeaa65-f001-48ce-b023-6d8ffd33fce7 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:38.435630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ee8b6-3519-4dd4-bb8b-052cc5c32b43 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:38.453653 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d153833-99b6-460e-880b-a2a85435e162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:38.454321 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d153833-99b6-460e-880b-a2a85435e162 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:38.454604 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d153833-99b6-460e-880b-a2a85435e162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:38.454871 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d153833-99b6-460e-880b-a2a85435e162 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:38.455536 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d153833-99b6-460e-880b-a2a85435e162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:38.457193 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d153833-99b6-460e-880b-a2a85435e162 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:38.460896 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a16ff5-5579-4985-90c7-688fd7eb27a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:38.461569 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a16ff5-5579-4985-90c7-688fd7eb27a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:38.461905 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a16ff5-5579-4985-90c7-688fd7eb27a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:38.462844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a16ff5-5579-4985-90c7-688fd7eb27a1 admin admin] PATH_INFO: `/v3/projects/fb059805c019400ea97c73dda8404254` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:38.463209 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:38.468029 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a16ff5-5579-4985-90c7-688fd7eb27a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb059805c019400ea97c73dda8404254)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:38.469677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a16ff5-5579-4985-90c7-688fd7eb27a1 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:38.519968 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 854/1703] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:38.522390 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d153833-99b6-460e-880b-a2a85435e162 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:38.544232 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d153833-99b6-460e-880b-a2a85435e162 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:38.562712 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50993c7e-7f10-4894-bbee-d217a6c713c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:38.563207 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50993c7e-7f10-4894-bbee-d217a6c713c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:38.563463 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50993c7e-7f10-4894-bbee-d217a6c713c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:38.563746 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50993c7e-7f10-4894-bbee-d217a6c713c7 admin admin] PATH_INFO: `/v3/projects/c24a675aa0974e878c5b49d1032af37e/users/06df6b9b3dd742c98afae7bb81f7076e/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:38.611965 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50993c7e-7f10-4894-bbee-d217a6c713c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c24a675aa0974e878c5b49d1032af37e, user_id=06df6b9b3dd742c98afae7bb81f7076e, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:38.616498 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50993c7e-7f10-4894-bbee-d217a6c713c7 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:38.618442 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-36a16ff5-5579-4985-90c7-688fd7eb27a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb059805c019400ea97c73dda8404254', 'request_id': 'req-36a16ff5-5579-4985-90c7-688fd7eb27a1'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:38.672286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-36a16ff5-5579-4985-90c7-688fd7eb27a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb059805c019400ea97c73dda8404254', 'request_id': 'req-36a16ff5-5579-4985-90c7-688fd7eb27a1'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:38.684362 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 855/1704] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:38 2022] PUT /identity/v3/projects/c24a675aa0974e878c5b49d1032af37e/users/06df6b9b3dd742c98afae7bb81f7076e/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:38.688913 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbba1c-de4a-4010-ab9e-24a0a4ffb709 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:38.693024 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbba1c-de4a-4010-ab9e-24a0a4ffb709 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:38.694588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbba1c-de4a-4010-ab9e-24a0a4ffb709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:38.710021 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-36a16ff5-5579-4985-90c7-688fd7eb27a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb059805c019400ea97c73dda8404254', 'request_id': 'req-36a16ff5-5579-4985-90c7-688fd7eb27a1'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:38.732687 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-32cbba1c-de4a-4010-ab9e-24a0a4ffb709 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:38.745455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-32cbba1c-de4a-4010-ab9e-24a0a4ffb709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:38.752022 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-36a16ff5-5579-4985-90c7-688fd7eb27a1 admin admin] The token cache is being invalidate because project fb059805c019400ea97c73dda8404254 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:12:38.752716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-36a16ff5-5579-4985-90c7-688fd7eb27a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:38.753737 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-36a16ff5-5579-4985-90c7-688fd7eb27a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:38.760103 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 850/1705] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:12:38 2022] DELETE /identity/v3/projects/fb059805c019400ea97c73dda8404254 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:38.761457 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a16ff5-5579-4985-90c7-688fd7eb27a1 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:38.765366 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-36a16ff5-5579-4985-90c7-688fd7eb27a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:38.859710 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 856/1706] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:38.861604 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e948fa-e415-4b9b-ad8a-3ad5bfdcc6af None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:38.863892 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e948fa-e415-4b9b-ad8a-3ad5bfdcc6af None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:38.863892 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e948fa-e415-4b9b-ad8a-3ad5bfdcc6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:38.918623 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-a2e948fa-e415-4b9b-ad8a-3ad5bfdcc6af None None] MFA Rules not processed for user `06df6b9b3dd742c98afae7bb81f7076e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:38.922523 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a2e948fa-e415-4b9b-ad8a-3ad5bfdcc6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:38.974128 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a16ff5-5579-4985-90c7-688fd7eb27a1 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:38.986657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c296a39-dae7-4c76-a356-9d0da19b5f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:38.989464 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c296a39-dae7-4c76-a356-9d0da19b5f61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:38.989757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c296a39-dae7-4c76-a356-9d0da19b5f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:38.990037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c296a39-dae7-4c76-a356-9d0da19b5f61 admin admin] PATH_INFO: `/v3/projects/6f4cd662f318463d8d8f0ea7a053b22e/users/746ad1d9ffdf4b3084e9a2044bf2aa48/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:39.033855 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c296a39-dae7-4c76-a356-9d0da19b5f61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f4cd662f318463d8d8f0ea7a053b22e, user_id=746ad1d9ffdf4b3084e9a2044bf2aa48, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:39.040927 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c296a39-dae7-4c76-a356-9d0da19b5f61 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:39.114551 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 851/1707] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:38 2022] PUT /identity/v3/projects/6f4cd662f318463d8d8f0ea7a053b22e/users/746ad1d9ffdf4b3084e9a2044bf2aa48/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:39.117960 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c296a39-dae7-4c76-a356-9d0da19b5f61 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:39.124837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-1c296a39-dae7-4c76-a356-9d0da19b5f61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:39.156355 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 857/1708] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:39.159518 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e948fa-e415-4b9b-ad8a-3ad5bfdcc6af None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:39.197316 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e948fa-e415-4b9b-ad8a-3ad5bfdcc6af None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:39.236885 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:39.307433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c296a39-dae7-4c76-a356-9d0da19b5f61 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:39.331720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb71f7-5f80-4c4b-98a7-5a86c599b627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:39.333601 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb71f7-5f80-4c4b-98a7-5a86c599b627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:39.334193 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb71f7-5f80-4c4b-98a7-5a86c599b627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:39.334472 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb71f7-5f80-4c4b-98a7-5a86c599b627 admin admin] PATH_INFO: `/v3/users/058e7536fa7d4c6abceba8c21079d01a` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:39.336565 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb71f7-5f80-4c4b-98a7-5a86c599b627 admin admin] RBAC: Authorizing `identity:delete_user(user_id=058e7536fa7d4c6abceba8c21079d01a)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:39.338897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb71f7-5f80-4c4b-98a7-5a86c599b627 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:39.355248 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10f1a8-d5f2-4156-8e19-f8e35161914d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:39.355896 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10f1a8-d5f2-4156-8e19-f8e35161914d admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:39.356174 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10f1a8-d5f2-4156-8e19-f8e35161914d admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:39.356446 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10f1a8-d5f2-4156-8e19-f8e35161914d admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:39.357685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10f1a8-d5f2-4156-8e19-f8e35161914d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:12:39.358010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10f1a8-d5f2-4156-8e19-f8e35161914d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:39.359930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10f1a8-d5f2-4156-8e19-f8e35161914d admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:39.377734 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:39.414522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 858/1709] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:12:39.414522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10f1a8-d5f2-4156-8e19-f8e35161914d admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:39.457127 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10f1a8-d5f2-4156-8e19-f8e35161914d admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:39.473744 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bea5ca-271a-47aa-ac86-b7f2ab1e5ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:39.473744 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bea5ca-271a-47aa-ac86-b7f2ab1e5ae7 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:39.476875 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bea5ca-271a-47aa-ac86-b7f2ab1e5ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:39.477210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bea5ca-271a-47aa-ac86-b7f2ab1e5ae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:39.479352 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bea5ca-271a-47aa-ac86-b7f2ab1e5ae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:39.488548 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bea5ca-271a-47aa-ac86-b7f2ab1e5ae7 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:39.493988 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-13fb71f7-5f80-4c4b-98a7-5a86c599b627 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '058e7536fa7d4c6abceba8c21079d01a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:39.517922 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-13fb71f7-5f80-4c4b-98a7-5a86c599b627 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '058e7536fa7d4c6abceba8c21079d01a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:39.528556 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 859/1710] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:39.530205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bea5ca-271a-47aa-ac86-b7f2ab1e5ae7 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:39.540543 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c0bea5ca-271a-47aa-ac86-b7f2ab1e5ae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:39.568289 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-13fb71f7-5f80-4c4b-98a7-5a86c599b627 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '058e7536fa7d4c6abceba8c21079d01a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:39.577290 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-13fb71f7-5f80-4c4b-98a7-5a86c599b627 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '058e7536fa7d4c6abceba8c21079d01a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:39.578270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-13fb71f7-5f80-4c4b-98a7-5a86c599b627 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '058e7536fa7d4c6abceba8c21079d01a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:39.590386 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 852/1711] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:12:39 2022] DELETE /identity/v3/users/058e7536fa7d4c6abceba8c21079d01a => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:39.592478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb71f7-5f80-4c4b-98a7-5a86c599b627 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:39.594200 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-13fb71f7-5f80-4c4b-98a7-5a86c599b627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:39.603717 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bea5ca-271a-47aa-ac86-b7f2ab1e5ae7 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:39.689229 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb71f7-5f80-4c4b-98a7-5a86c599b627 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:39.695346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920f7df-d7d9-4c18-9018-5537e09c03bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:39.695991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920f7df-d7d9-4c18-9018-5537e09c03bc admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:39.696274 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920f7df-d7d9-4c18-9018-5537e09c03bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:39.696544 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920f7df-d7d9-4c18-9018-5537e09c03bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:39.697296 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920f7df-d7d9-4c18-9018-5537e09c03bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:39.699954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920f7df-d7d9-4c18-9018-5537e09c03bc admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:39.793185 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 853/1712] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:39 2022] POST /identity/v3/projects => generated 365 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:39.798084 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920f7df-d7d9-4c18-9018-5537e09c03bc admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:39.802975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d920f7df-d7d9-4c18-9018-5537e09c03bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:39.821034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715ba39-6290-461d-8249-b2d7bf55cc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:39.821832 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715ba39-6290-461d-8249-b2d7bf55cc0f admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:39.822024 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715ba39-6290-461d-8249-b2d7bf55cc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:39.822262 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715ba39-6290-461d-8249-b2d7bf55cc0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:39.822899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715ba39-6290-461d-8249-b2d7bf55cc0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:39.824815 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715ba39-6290-461d-8249-b2d7bf55cc0f admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:39.890759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 860/1713] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:39 2022] POST /identity/v3/projects => generated 371 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:39.891878 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715ba39-6290-461d-8249-b2d7bf55cc0f admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:39.897634 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-6715ba39-6290-461d-8249-b2d7bf55cc0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:39.955784 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:39.991650 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920f7df-d7d9-4c18-9018-5537e09c03bc admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:40.080441 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715ba39-6290-461d-8249-b2d7bf55cc0f admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:40.096409 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2165e5-4e86-4700-9324-f47979aed15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:40.097469 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2165e5-4e86-4700-9324-f47979aed15a admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:40.098163 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2165e5-4e86-4700-9324-f47979aed15a admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:40.101969 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2165e5-4e86-4700-9324-f47979aed15a admin admin] PATH_INFO: `/v3/projects/6f4cd662f318463d8d8f0ea7a053b22e/users/746ad1d9ffdf4b3084e9a2044bf2aa48/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:40.104836 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ebfe4-4939-43e1-9903-01bb794ee8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:40.105005 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ebfe4-4939-43e1-9903-01bb794ee8f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:40.105145 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ebfe4-4939-43e1-9903-01bb794ee8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:40.105421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ebfe4-4939-43e1-9903-01bb794ee8f5 admin admin] PATH_INFO: `/v3/users/3622f140137340e985d256a64fc360e6` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:40.107486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ebfe4-4939-43e1-9903-01bb794ee8f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3622f140137340e985d256a64fc360e6)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:40.111110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ebfe4-4939-43e1-9903-01bb794ee8f5 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:40.111772 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2165e5-4e86-4700-9324-f47979aed15a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f4cd662f318463d8d8f0ea7a053b22e, user_id=746ad1d9ffdf4b3084e9a2044bf2aa48, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:40.125465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2165e5-4e86-4700-9324-f47979aed15a admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:40.150445 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 854/1714] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:39 2022] PUT /identity/v3/projects/6f4cd662f318463d8d8f0ea7a053b22e/users/746ad1d9ffdf4b3084e9a2044bf2aa48/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:40.151708 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2165e5-4e86-4700-9324-f47979aed15a admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:40.210431 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8f4ebfe4-4939-43e1-9903-01bb794ee8f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3622f140137340e985d256a64fc360e6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:40.251244 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8f4ebfe4-4939-43e1-9903-01bb794ee8f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3622f140137340e985d256a64fc360e6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:40.270099 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8f4ebfe4-4939-43e1-9903-01bb794ee8f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3622f140137340e985d256a64fc360e6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:40.272042 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:40.279699 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2165e5-4e86-4700-9324-f47979aed15a admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:40.288359 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8f4ebfe4-4939-43e1-9903-01bb794ee8f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3622f140137340e985d256a64fc360e6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:40.289179 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681c101-5879-422a-b02c-caae082865b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:40.289665 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8f4ebfe4-4939-43e1-9903-01bb794ee8f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3622f140137340e985d256a64fc360e6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:40.290390 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681c101-5879-422a-b02c-caae082865b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:40.290808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681c101-5879-422a-b02c-caae082865b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:40.291380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681c101-5879-422a-b02c-caae082865b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:40.292425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681c101-5879-422a-b02c-caae082865b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:40.297465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681c101-5879-422a-b02c-caae082865b8 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:40.304542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 861/1715] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:12:39 2022] DELETE /identity/v3/users/3622f140137340e985d256a64fc360e6 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:40.305592 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ebfe4-4939-43e1-9903-01bb794ee8f5 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:40.309556 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-8f4ebfe4-4939-43e1-9903-01bb794ee8f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:40.341903 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 855/1716] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:40 2022] POST /identity/v3/users => generated 283 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:40.346298 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ff84a-0c92-49d8-8124-f7c8c9991930 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:40.346959 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ff84a-0c92-49d8-8124-f7c8c9991930 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:40.347709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ff84a-0c92-49d8-8124-f7c8c9991930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:40.423184 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-819ff84a-0c92-49d8-8124-f7c8c9991930 None None] MFA Rules not processed for user `746ad1d9ffdf4b3084e9a2044bf2aa48`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:40.424765 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-819ff84a-0c92-49d8-8124-f7c8c9991930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:40.470266 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ebfe4-4939-43e1-9903-01bb794ee8f5 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:40.475975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77886676-2b87-4d6a-8461-663d2425658c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:40.477375 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77886676-2b87-4d6a-8461-663d2425658c admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:40.477900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77886676-2b87-4d6a-8461-663d2425658c admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:40.478481 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77886676-2b87-4d6a-8461-663d2425658c admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:40.479310 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77886676-2b87-4d6a-8461-663d2425658c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:40.482518 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77886676-2b87-4d6a-8461-663d2425658c admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:40.518288 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 862/1717] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:40 2022] POST /identity/v3/users => generated 292 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:40.522310 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a659538-a130-4d16-8d1d-93e1d0564979 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:40.522743 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a659538-a130-4d16-8d1d-93e1d0564979 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:40.523214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a659538-a130-4d16-8d1d-93e1d0564979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:40.565796 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-6a659538-a130-4d16-8d1d-93e1d0564979 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:40.567883 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-6a659538-a130-4d16-8d1d-93e1d0564979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:40.581941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 863/1718] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:40.583405 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a659538-a130-4d16-8d1d-93e1d0564979 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:40.587845 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:40.589466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-6a659538-a130-4d16-8d1d-93e1d0564979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:40.609204 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 856/1719] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:40.609204 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ff84a-0c92-49d8-8124-f7c8c9991930 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:40.626293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a659538-a130-4d16-8d1d-93e1d0564979 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:40.630752 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ff84a-0c92-49d8-8124-f7c8c9991930 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:40.633123 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b9cfc-15da-49cf-b76a-534e523ee383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:40.634016 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b9cfc-15da-49cf-b76a-534e523ee383 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:40.634483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b9cfc-15da-49cf-b76a-534e523ee383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:40.634757 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b9cfc-15da-49cf-b76a-534e523ee383 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:40.635512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b9cfc-15da-49cf-b76a-534e523ee383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:40.639085 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b9cfc-15da-49cf-b76a-534e523ee383 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:40.642526 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7461ae-8b35-4b7a-8d7d-98c57b935ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:40.643189 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7461ae-8b35-4b7a-8d7d-98c57b935ddf admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:40.643470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7461ae-8b35-4b7a-8d7d-98c57b935ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:40.643745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7461ae-8b35-4b7a-8d7d-98c57b935ddf admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:40.644678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7461ae-8b35-4b7a-8d7d-98c57b935ddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:40.646435 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7461ae-8b35-4b7a-8d7d-98c57b935ddf admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:40.669849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 864/1720] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:40.671345 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 857/1721] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:40.672292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7461ae-8b35-4b7a-8d7d-98c57b935ddf admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:40.682575 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b9cfc-15da-49cf-b76a-534e523ee383 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:40.685032 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a53b9cfc-15da-49cf-b76a-534e523ee383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:40.686919 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-8e7461ae-8b35-4b7a-8d7d-98c57b935ddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:40.795689 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b9cfc-15da-49cf-b76a-534e523ee383 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:40.809871 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf30186-2ef8-4748-a867-9752354b1225 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:40.812612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf30186-2ef8-4748-a867-9752354b1225 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:40.813087 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf30186-2ef8-4748-a867-9752354b1225 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:40.813366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf30186-2ef8-4748-a867-9752354b1225 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:40.825322 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf30186-2ef8-4748-a867-9752354b1225 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:40.826934 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf30186-2ef8-4748-a867-9752354b1225 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:40.829114 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7461ae-8b35-4b7a-8d7d-98c57b935ddf admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:40.842695 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a12b02-de77-49ff-94a8-b37cd8c737be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:40.843342 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a12b02-de77-49ff-94a8-b37cd8c737be service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:40.843624 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a12b02-de77-49ff-94a8-b37cd8c737be service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:40.843899 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a12b02-de77-49ff-94a8-b37cd8c737be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:40.848308 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 865/1722] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:40.849438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf30186-2ef8-4748-a867-9752354b1225 service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:40.869207 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf30186-2ef8-4748-a867-9752354b1225 service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:40.880675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a12b02-de77-49ff-94a8-b37cd8c737be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:40.882309 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a12b02-de77-49ff-94a8-b37cd8c737be service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:40.883534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a8f50-05df-40d5-81c0-3c0b9c7a032b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:40.884555 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a8f50-05df-40d5-81c0-3c0b9c7a032b admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:40.885003 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a8f50-05df-40d5-81c0-3c0b9c7a032b admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:40.885415 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a8f50-05df-40d5-81c0-3c0b9c7a032b admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:40.888917 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a8f50-05df-40d5-81c0-3c0b9c7a032b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:40.891987 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a8f50-05df-40d5-81c0-3c0b9c7a032b admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:40.915911 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 858/1723] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:40 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:40.916778 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a12b02-de77-49ff-94a8-b37cd8c737be service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:40.933108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 866/1724] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:40.934559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a8f50-05df-40d5-81c0-3c0b9c7a032b admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:40.954547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a8f50-05df-40d5-81c0-3c0b9c7a032b admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:40.958937 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a12b02-de77-49ff-94a8-b37cd8c737be service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:40.968301 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba56bbf-b7e4-4854-979e-047092996963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:40.969113 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba56bbf-b7e4-4854-979e-047092996963 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:40.969341 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba56bbf-b7e4-4854-979e-047092996963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:40.969559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d19447-837b-4f34-b4af-ac11bfc2c53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:40.969667 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba56bbf-b7e4-4854-979e-047092996963 admin admin] PATH_INFO: `/v3/projects/94e7f861437f4633a71bed1c275d6897/users/0de6bd593699495294a7def9a7b8be9b/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:40.970767 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d19447-837b-4f34-b4af-ac11bfc2c53f admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:40.971259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d19447-837b-4f34-b4af-ac11bfc2c53f admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:40.971745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d19447-837b-4f34-b4af-ac11bfc2c53f admin admin] PATH_INFO: `/v3/projects/aad2e0d8f0da4a6b8feade51237cecf2/users/0c77992702bc4c28af464efb75df5aa6/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:41.012094 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba56bbf-b7e4-4854-979e-047092996963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94e7f861437f4633a71bed1c275d6897, user_id=0de6bd593699495294a7def9a7b8be9b, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:41.020582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d19447-837b-4f34-b4af-ac11bfc2c53f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad2e0d8f0da4a6b8feade51237cecf2, user_id=0c77992702bc4c28af464efb75df5aa6, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:41.022721 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba56bbf-b7e4-4854-979e-047092996963 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:41.026262 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d19447-837b-4f34-b4af-ac11bfc2c53f admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:41.064351 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 867/1725] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:40 2022] PUT /identity/v3/projects/94e7f861437f4633a71bed1c275d6897/users/0de6bd593699495294a7def9a7b8be9b/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:41.077745 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebde5c-9c78-44b8-b334-92c983407cf9 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:41.078904 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebde5c-9c78-44b8-b334-92c983407cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:41.080548 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebde5c-9c78-44b8-b334-92c983407cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:41.115914 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 859/1726] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:40 2022] PUT /identity/v3/projects/aad2e0d8f0da4a6b8feade51237cecf2/users/0c77992702bc4c28af464efb75df5aa6/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:41.120570 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d19447-837b-4f34-b4af-ac11bfc2c53f admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:41.190511 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-afebde5c-9c78-44b8-b334-92c983407cf9 None None] MFA Rules not processed for user `0de6bd593699495294a7def9a7b8be9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:41.194363 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-afebde5c-9c78-44b8-b334-92c983407cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:41.243385 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:41.344099 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 868/1727] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:41.345109 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afebde5c-9c78-44b8-b334-92c983407cf9 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:41.349732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-afebde5c-9c78-44b8-b334-92c983407cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:41.365599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d19447-837b-4f34-b4af-ac11bfc2c53f admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:41.380565 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011613a3-9688-4f54-bce6-eb061096302c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:41.385082 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011613a3-9688-4f54-bce6-eb061096302c admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:41.386767 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afebde5c-9c78-44b8-b334-92c983407cf9 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:41.390977 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011613a3-9688-4f54-bce6-eb061096302c admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:41.392353 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011613a3-9688-4f54-bce6-eb061096302c admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:41.393586 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011613a3-9688-4f54-bce6-eb061096302c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:41.397280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70af7a6-4db8-49ef-ae32-69283b2901ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:41.399440 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70af7a6-4db8-49ef-ae32-69283b2901ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:41.399769 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011613a3-9688-4f54-bce6-eb061096302c admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:41.400212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70af7a6-4db8-49ef-ae32-69283b2901ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:41.400596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70af7a6-4db8-49ef-ae32-69283b2901ac admin admin] PATH_INFO: `/v3/projects/8bc0b25f4adb4535828370a0cac25577` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:41.409489 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70af7a6-4db8-49ef-ae32-69283b2901ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bc0b25f4adb4535828370a0cac25577)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:41.418738 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70af7a6-4db8-49ef-ae32-69283b2901ac admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:41.456717 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 860/1728] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:12:41.460201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011613a3-9688-4f54-bce6-eb061096302c admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:41.495373 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011613a3-9688-4f54-bce6-eb061096302c admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:41.523717 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a70af7a6-4db8-49ef-ae32-69283b2901ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bc0b25f4adb4535828370a0cac25577', 'request_id': 'req-a70af7a6-4db8-49ef-ae32-69283b2901ac'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:41.534483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a70af7a6-4db8-49ef-ae32-69283b2901ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bc0b25f4adb4535828370a0cac25577', 'request_id': 'req-a70af7a6-4db8-49ef-ae32-69283b2901ac'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:41.555331 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a70af7a6-4db8-49ef-ae32-69283b2901ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bc0b25f4adb4535828370a0cac25577', 'request_id': 'req-a70af7a6-4db8-49ef-ae32-69283b2901ac'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:41.588735 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a70af7a6-4db8-49ef-ae32-69283b2901ac admin admin] The token cache is being invalidate because project 8bc0b25f4adb4535828370a0cac25577 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:12:41.589767 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a70af7a6-4db8-49ef-ae32-69283b2901ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:41.592015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a70af7a6-4db8-49ef-ae32-69283b2901ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:41.598530 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 869/1729] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:12:41 2022] DELETE /identity/v3/projects/8bc0b25f4adb4535828370a0cac25577 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:41.598530 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70af7a6-4db8-49ef-ae32-69283b2901ac admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:41.602828 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a70af7a6-4db8-49ef-ae32-69283b2901ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:41.726794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bc7b9-93f8-499a-8c32-88bece7937c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:41.727525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bc7b9-93f8-499a-8c32-88bece7937c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:41.728322 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bc7b9-93f8-499a-8c32-88bece7937c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:41.728322 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bc7b9-93f8-499a-8c32-88bece7937c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:41.729158 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369bc7b9-93f8-499a-8c32-88bece7937c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:41.730955 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369bc7b9-93f8-499a-8c32-88bece7937c5 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:41.737740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:41.750995 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70af7a6-4db8-49ef-ae32-69283b2901ac admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:41.768679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d62849-ccd5-4dfc-b4d8-fdc8b96ff531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:41.769777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d62849-ccd5-4dfc-b4d8-fdc8b96ff531 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:41.770063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d62849-ccd5-4dfc-b4d8-fdc8b96ff531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:41.770340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d62849-ccd5-4dfc-b4d8-fdc8b96ff531 admin admin] PATH_INFO: `/v3/projects/aad2e0d8f0da4a6b8feade51237cecf2/users/0c77992702bc4c28af464efb75df5aa6/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:41.778256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d62849-ccd5-4dfc-b4d8-fdc8b96ff531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad2e0d8f0da4a6b8feade51237cecf2, user_id=0c77992702bc4c28af464efb75df5aa6, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:41.820457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d62849-ccd5-4dfc-b4d8-fdc8b96ff531 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:41.832343 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 861/1730] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:41 2022] POST /identity/v3/projects => generated 363 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:41.837583 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc7be2-8658-4c12-8838-b42db13cc799 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:41.838009 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc7be2-8658-4c12-8838-b42db13cc799 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:41.838284 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc7be2-8658-4c12-8838-b42db13cc799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:41.884516 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 870/1731] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:41 2022] PUT /identity/v3/projects/aad2e0d8f0da4a6b8feade51237cecf2/users/0c77992702bc4c28af464efb75df5aa6/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:41.886834 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d62849-ccd5-4dfc-b4d8-fdc8b96ff531 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:41.890835 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-87d62849-ccd5-4dfc-b4d8-fdc8b96ff531 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:41.911089 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-cdcc7be2-8658-4c12-8838-b42db13cc799 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:41.918866 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-cdcc7be2-8658-4c12-8838-b42db13cc799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:41.994346 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:42.081458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d62849-ccd5-4dfc-b4d8-fdc8b96ff531 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:42.088961 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 862/1732] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:42.091594 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46666f77-c371-493d-b224-871b64a0df0d None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:42.091971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46666f77-c371-493d-b224-871b64a0df0d None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:42.092265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46666f77-c371-493d-b224-871b64a0df0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:42.092824 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c37a98-c726-4158-b489-94281c6102b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:42.093470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c37a98-c726-4158-b489-94281c6102b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:42.093779 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c37a98-c726-4158-b489-94281c6102b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:42.094075 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c37a98-c726-4158-b489-94281c6102b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:42.094713 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c37a98-c726-4158-b489-94281c6102b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:42.101679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c37a98-c726-4158-b489-94281c6102b1 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:42.146650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 871/1733] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:41 2022] POST /identity/v3/users => generated 288 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:42.147830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c37a98-c726-4158-b489-94281c6102b1 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:42.150532 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-46666f77-c371-493d-b224-871b64a0df0d None None] MFA Rules not processed for user `0c77992702bc4c28af464efb75df5aa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:42.152679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-46666f77-c371-493d-b224-871b64a0df0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:42.155104 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-71c37a98-c726-4158-b489-94281c6102b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:42.182420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c37a98-c726-4158-b489-94281c6102b1 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:42.196462 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef73a5-75e6-4d64-b7a4-110913d03ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:42.197524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef73a5-75e6-4d64-b7a4-110913d03ef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:42.198655 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef73a5-75e6-4d64-b7a4-110913d03ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:42.199263 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef73a5-75e6-4d64-b7a4-110913d03ef2 admin admin] PATH_INFO: `/v3/users/3056b78f7d7f4c3e9bcba1a420990e44` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:42.201076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef73a5-75e6-4d64-b7a4-110913d03ef2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3056b78f7d7f4c3e9bcba1a420990e44)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:42.204447 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef73a5-75e6-4d64-b7a4-110913d03ef2 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:42.301759 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 863/1734] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:42 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:42.304089 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc0fa1-57c0-438e-8b2e-1916b87946b8 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:42.304550 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc0fa1-57c0-438e-8b2e-1916b87946b8 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:42.305206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc0fa1-57c0-438e-8b2e-1916b87946b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:42.316053 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-b6ef73a5-75e6-4d64-b7a4-110913d03ef2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3056b78f7d7f4c3e9bcba1a420990e44'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:42.336358 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-b6ef73a5-75e6-4d64-b7a4-110913d03ef2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3056b78f7d7f4c3e9bcba1a420990e44'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:42.346032 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-20dc0fa1-57c0-438e-8b2e-1916b87946b8 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:42.347326 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-20dc0fa1-57c0-438e-8b2e-1916b87946b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:42.351150 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-b6ef73a5-75e6-4d64-b7a4-110913d03ef2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3056b78f7d7f4c3e9bcba1a420990e44'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:42.363350 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 864/1735] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:42.365198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-b6ef73a5-75e6-4d64-b7a4-110913d03ef2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3056b78f7d7f4c3e9bcba1a420990e44'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:42.366190 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-b6ef73a5-75e6-4d64-b7a4-110913d03ef2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3056b78f7d7f4c3e9bcba1a420990e44'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:42.369718 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8109c6-d71b-4225-9ed7-ff6a786a58ed None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:42.370467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8109c6-d71b-4225-9ed7-ff6a786a58ed None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:42.370739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8109c6-d71b-4225-9ed7-ff6a786a58ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:42.381356 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 872/1736] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:12:42 2022] DELETE /identity/v3/users/3056b78f7d7f4c3e9bcba1a420990e44 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:42.382811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef73a5-75e6-4d64-b7a4-110913d03ef2 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:42.385994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-b6ef73a5-75e6-4d64-b7a4-110913d03ef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:42.433009 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-1c8109c6-d71b-4225-9ed7-ff6a786a58ed None None] MFA Rules not processed for user `04c2aae2da284df29243b99d33725ddd`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:42.436793 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-1c8109c6-d71b-4225-9ed7-ff6a786a58ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:42.517513 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef73a5-75e6-4d64-b7a4-110913d03ef2 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:42.529774 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67903a9e-86c2-4ea7-bf68-4614965196fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:42.530619 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:42.532823 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67903a9e-86c2-4ea7-bf68-4614965196fe admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:42.537487 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67903a9e-86c2-4ea7-bf68-4614965196fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:42.541020 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67903a9e-86c2-4ea7-bf68-4614965196fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:42.544632 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67903a9e-86c2-4ea7-bf68-4614965196fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:42.548131 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 865/1737] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:42.549156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8109c6-d71b-4225-9ed7-ff6a786a58ed None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:42.549607 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67903a9e-86c2-4ea7-bf68-4614965196fe admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:42.571529 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-1c8109c6-d71b-4225-9ed7-ff6a786a58ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:42.576735 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 873/1738] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:42.582633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67903a9e-86c2-4ea7-bf68-4614965196fe admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:42.588590 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-67903a9e-86c2-4ea7-bf68-4614965196fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:42.592513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8109c6-d71b-4225-9ed7-ff6a786a58ed None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:42.598973 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd6eec-c979-4c06-9c00-6e85616030e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:42.600978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd6eec-c979-4c06-9c00-6e85616030e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:42.601270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd6eec-c979-4c06-9c00-6e85616030e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:42.601549 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd6eec-c979-4c06-9c00-6e85616030e9 admin admin] PATH_INFO: `/v3/projects/b644d63d813746bfa8dc52f42973aba0` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:42.603366 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd6eec-c979-4c06-9c00-6e85616030e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b644d63d813746bfa8dc52f42973aba0)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:42.605311 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd6eec-c979-4c06-9c00-6e85616030e9 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:42.669098 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:42.698571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d0dd6eec-c979-4c06-9c00-6e85616030e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b644d63d813746bfa8dc52f42973aba0', 'request_id': 'req-d0dd6eec-c979-4c06-9c00-6e85616030e9'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:42.709493 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d0dd6eec-c979-4c06-9c00-6e85616030e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b644d63d813746bfa8dc52f42973aba0', 'request_id': 'req-d0dd6eec-c979-4c06-9c00-6e85616030e9'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:42.719427 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d0dd6eec-c979-4c06-9c00-6e85616030e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b644d63d813746bfa8dc52f42973aba0', 'request_id': 'req-d0dd6eec-c979-4c06-9c00-6e85616030e9'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:42.736810 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d0dd6eec-c979-4c06-9c00-6e85616030e9 admin admin] The token cache is being invalidate because project b644d63d813746bfa8dc52f42973aba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:12:42.737115 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d0dd6eec-c979-4c06-9c00-6e85616030e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:42.737684 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67903a9e-86c2-4ea7-bf68-4614965196fe admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:42.737988 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d0dd6eec-c979-4c06-9c00-6e85616030e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:42.740626 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 866/1739] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:12:42 2022] DELETE /identity/v3/projects/b644d63d813746bfa8dc52f42973aba0 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:42.741121 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd6eec-c979-4c06-9c00-6e85616030e9 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:42.743192 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d0dd6eec-c979-4c06-9c00-6e85616030e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:42.830253 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7e1a3-37d4-4855-beff-ac16ff87a3b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:42.830957 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7e1a3-37d4-4855-beff-ac16ff87a3b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:42.831145 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7e1a3-37d4-4855-beff-ac16ff87a3b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:42.832926 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7e1a3-37d4-4855-beff-ac16ff87a3b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:42.835305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-22f7e1a3-37d4-4855-beff-ac16ff87a3b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:42.851639 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd6eec-c979-4c06-9c00-6e85616030e9 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:42.904598 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c007326-0efb-458f-b94b-c2a35636ca9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:42.905261 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c007326-0efb-458f-b94b-c2a35636ca9f admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:42.905545 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c007326-0efb-458f-b94b-c2a35636ca9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:42.906402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c007326-0efb-458f-b94b-c2a35636ca9f admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:42.907794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c007326-0efb-458f-b94b-c2a35636ca9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:12:42.908112 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c007326-0efb-458f-b94b-c2a35636ca9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:42.911518 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c007326-0efb-458f-b94b-c2a35636ca9f admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:42.928339 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 867/1740] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:12:42.929792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd976a46-5ede-4b0e-8d7b-0d703c6d9ecf None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:42.930087 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd976a46-5ede-4b0e-8d7b-0d703c6d9ecf None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:42.930368 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd976a46-5ede-4b0e-8d7b-0d703c6d9ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:42.982884 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-fd976a46-5ede-4b0e-8d7b-0d703c6d9ecf None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:42.984329 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-fd976a46-5ede-4b0e-8d7b-0d703c6d9ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:42.998534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 868/1741] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:42.998534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd976a46-5ede-4b0e-8d7b-0d703c6d9ecf None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:43.002173 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-fd976a46-5ede-4b0e-8d7b-0d703c6d9ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:43.006357 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f7e1a3-37d4-4855-beff-ac16ff87a3b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:43.007845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f7e1a3-37d4-4855-beff-ac16ff87a3b5 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:43.042546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 874/1742] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:42 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:43.043995 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7e1a3-37d4-4855-beff-ac16ff87a3b5 service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:43.047603 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-22f7e1a3-37d4-4855-beff-ac16ff87a3b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:43.084986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7e1a3-37d4-4855-beff-ac16ff87a3b5 service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:43.099505 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91017ab-81d7-46b8-8d20-c797f8ac4fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:43.100973 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91017ab-81d7-46b8-8d20-c797f8ac4fb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:43.102555 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91017ab-81d7-46b8-8d20-c797f8ac4fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:43.102945 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91017ab-81d7-46b8-8d20-c797f8ac4fb0 admin admin] PATH_INFO: `/v3/projects/a1a0c81cfa1042d49e2b6b11690e63db/users/a05f43e064124f518c4b614586ec2c4f/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:43.129764 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91017ab-81d7-46b8-8d20-c797f8ac4fb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a0c81cfa1042d49e2b6b11690e63db, user_id=a05f43e064124f518c4b614586ec2c4f, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:43.143531 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91017ab-81d7-46b8-8d20-c797f8ac4fb0 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:43.172614 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 875/1743] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:43 2022] PUT /identity/v3/projects/a1a0c81cfa1042d49e2b6b11690e63db/users/a05f43e064124f518c4b614586ec2c4f/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:43.177412 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91017ab-81d7-46b8-8d20-c797f8ac4fb0 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:43.179503 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c91017ab-81d7-46b8-8d20-c797f8ac4fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:43.197490 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:43.205216 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd976a46-5ede-4b0e-8d7b-0d703c6d9ecf None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:43.340790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91017ab-81d7-46b8-8d20-c797f8ac4fb0 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:43.350233 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c239e-bf35-454a-a937-e8a7c2099117 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:43.351022 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c239e-bf35-454a-a937-e8a7c2099117 service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:43.351420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c239e-bf35-454a-a937-e8a7c2099117 service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:43.351787 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c239e-bf35-454a-a937-e8a7c2099117 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:43.355408 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-7a4c239e-bf35-454a-a937-e8a7c2099117 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:43.364512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb53578-8c6f-424b-b826-ac0c28369907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:43.365078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb53578-8c6f-424b-b826-ac0c28369907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:43.365358 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb53578-8c6f-424b-b826-ac0c28369907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:43.365791 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb53578-8c6f-424b-b826-ac0c28369907 admin admin] PATH_INFO: `/v3/users/f30daa72513644bba09bf11ffaad3c87` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:43.368150 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb53578-8c6f-424b-b826-ac0c28369907 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f30daa72513644bba09bf11ffaad3c87)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:43.370430 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb53578-8c6f-424b-b826-ac0c28369907 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:43.411726 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:43.472956 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-edb53578-8c6f-424b-b826-ac0c28369907 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f30daa72513644bba09bf11ffaad3c87'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:43.479743 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c239e-bf35-454a-a937-e8a7c2099117 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:43.483325 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c239e-bf35-454a-a937-e8a7c2099117 service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:43.490236 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-edb53578-8c6f-424b-b826-ac0c28369907 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f30daa72513644bba09bf11ffaad3c87'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:43.503815 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 869/1744] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:43.504100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-edb53578-8c6f-424b-b826-ac0c28369907 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f30daa72513644bba09bf11ffaad3c87'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:43.505675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c239e-bf35-454a-a937-e8a7c2099117 service nova] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:43.515563 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-edb53578-8c6f-424b-b826-ac0c28369907 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f30daa72513644bba09bf11ffaad3c87'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:43.516399 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-edb53578-8c6f-424b-b826-ac0c28369907 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f30daa72513644bba09bf11ffaad3c87'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:43.524716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c239e-bf35-454a-a937-e8a7c2099117 service nova] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:43.535366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 876/1745] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:12:43 2022] DELETE /identity/v3/users/f30daa72513644bba09bf11ffaad3c87 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:43.536527 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb53578-8c6f-424b-b826-ac0c28369907 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:43.538111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-edb53578-8c6f-424b-b826-ac0c28369907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:43.556085 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c75160-7a13-4d57-a25c-a2db1ae1840f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:43.556731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c75160-7a13-4d57-a25c-a2db1ae1840f admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:43.557011 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c75160-7a13-4d57-a25c-a2db1ae1840f admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:43.557281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c75160-7a13-4d57-a25c-a2db1ae1840f admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:43.557974 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c75160-7a13-4d57-a25c-a2db1ae1840f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:43.581092 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c75160-7a13-4d57-a25c-a2db1ae1840f admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:43.679940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 870/1746] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:43 2022] POST /identity/v3/projects => generated 391 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:43.680905 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c75160-7a13-4d57-a25c-a2db1ae1840f admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:43.692881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-14c75160-7a13-4d57-a25c-a2db1ae1840f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:43.742589 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb53578-8c6f-424b-b826-ac0c28369907 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:43.871151 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c75160-7a13-4d57-a25c-a2db1ae1840f admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:43.887337 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a19ba-f70a-4319-bd16-3ab6fee634d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:43.888012 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a19ba-f70a-4319-bd16-3ab6fee634d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:43.888293 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a19ba-f70a-4319-bd16-3ab6fee634d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:43.888571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a19ba-f70a-4319-bd16-3ab6fee634d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:43.898820 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-222a19ba-f70a-4319-bd16-3ab6fee634d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:43.918743 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d640a-9198-4e70-abc7-df5038b69aba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:43.920517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d640a-9198-4e70-abc7-df5038b69aba service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:43.921274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d640a-9198-4e70-abc7-df5038b69aba service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:43.922118 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d640a-9198-4e70-abc7-df5038b69aba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:43.937623 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-6a2d640a-9198-4e70-abc7-df5038b69aba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:44.013221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:44.130781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a19ba-f70a-4319-bd16-3ab6fee634d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:44.132633 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a19ba-f70a-4319-bd16-3ab6fee634d0 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:44.155869 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 871/1747] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:44.177568 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a19ba-f70a-4319-bd16-3ab6fee634d0 service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:44.182945 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-222a19ba-f70a-4319-bd16-3ab6fee634d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:44.232997 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a19ba-f70a-4319-bd16-3ab6fee634d0 service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:44.234616 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d640a-9198-4e70-abc7-df5038b69aba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:44.246457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d640a-9198-4e70-abc7-df5038b69aba service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:44.251266 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594998e-9179-43e8-a483-44616cece83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:44.251959 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594998e-9179-43e8-a483-44616cece83b admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:44.252309 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594998e-9179-43e8-a483-44616cece83b admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:44.252656 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594998e-9179-43e8-a483-44616cece83b admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:44.253380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594998e-9179-43e8-a483-44616cece83b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:44.257324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594998e-9179-43e8-a483-44616cece83b admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:44.290839 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 872/1748] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:44.290839 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594998e-9179-43e8-a483-44616cece83b admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:44.295875 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-8594998e-9179-43e8-a483-44616cece83b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:44.325366 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594998e-9179-43e8-a483-44616cece83b admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:44.339686 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:44.339963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d4709-81b6-4834-876f-2bcf4a7b65be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:44.340469 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d4709-81b6-4834-876f-2bcf4a7b65be admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:44.340904 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d4709-81b6-4834-876f-2bcf4a7b65be admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:44.341257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d4709-81b6-4834-876f-2bcf4a7b65be admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:44.342094 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d4709-81b6-4834-876f-2bcf4a7b65be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:44.347717 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d4709-81b6-4834-876f-2bcf4a7b65be admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:44.376613 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 877/1749] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 840 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:44.380693 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68db5a-430f-4ad7-bd6f-7a2c0dba9065 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:44.381238 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68db5a-430f-4ad7-bd6f-7a2c0dba9065 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:44.381747 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68db5a-430f-4ad7-bd6f-7a2c0dba9065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:44.418283 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 873/1750] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:44 2022] POST /identity/v3/users => generated 296 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:44.419979 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d4709-81b6-4834-876f-2bcf4a7b65be admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:44.445424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d4709-81b6-4834-876f-2bcf4a7b65be admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:44.463488 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e8198-a331-441e-976d-fb716ff1954e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:44.464281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e8198-a331-441e-976d-fb716ff1954e admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:44.464638 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e8198-a331-441e-976d-fb716ff1954e admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:44.464981 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e8198-a331-441e-976d-fb716ff1954e admin admin] PATH_INFO: `/v3/projects/a1a0c81cfa1042d49e2b6b11690e63db/users/a05f43e064124f518c4b614586ec2c4f/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:44.470113 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5e8198-a331-441e-976d-fb716ff1954e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a0c81cfa1042d49e2b6b11690e63db, user_id=a05f43e064124f518c4b614586ec2c4f, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:44.473069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5e8198-a331-441e-976d-fb716ff1954e admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:44.517891 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-4e68db5a-430f-4ad7-bd6f-7a2c0dba9065 None None] MFA Rules not processed for user `e75b537d44524de192282b10825293f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:44.520048 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 874/1751] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:44 2022] PUT /identity/v3/projects/a1a0c81cfa1042d49e2b6b11690e63db/users/a05f43e064124f518c4b614586ec2c4f/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:44.521354 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e8198-a331-441e-976d-fb716ff1954e admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:44.525146 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-4e68db5a-430f-4ad7-bd6f-7a2c0dba9065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:44.687568 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e8198-a331-441e-976d-fb716ff1954e admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:44.690506 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 878/1752] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:44.692435 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c59a1-266c-4aa8-a122-78488298562e None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:44.692942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c59a1-266c-4aa8-a122-78488298562e None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:44.693393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c59a1-266c-4aa8-a122-78488298562e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:44.706836 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91521c8d-4149-4b1f-94e5-538c51dcf6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:44.707885 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91521c8d-4149-4b1f-94e5-538c51dcf6a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:44.709092 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91521c8d-4149-4b1f-94e5-538c51dcf6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:44.709092 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91521c8d-4149-4b1f-94e5-538c51dcf6a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:44.709288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91521c8d-4149-4b1f-94e5-538c51dcf6a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:44.710989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91521c8d-4149-4b1f-94e5-538c51dcf6a5 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:44.735745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 875/1753] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:44.739004 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175c552-4bb3-45da-8b7e-b9d13e3b494e None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:44.739004 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175c552-4bb3-45da-8b7e-b9d13e3b494e None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:44.740787 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175c552-4bb3-45da-8b7e-b9d13e3b494e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:44.770074 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-a57c59a1-266c-4aa8-a122-78488298562e None None] MFA Rules not processed for user `a05f43e064124f518c4b614586ec2c4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:44.778342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a57c59a1-266c-4aa8-a122-78488298562e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:44.831738 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-5175c552-4bb3-45da-8b7e-b9d13e3b494e None None] MFA Rules not processed for user `e75b537d44524de192282b10825293f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:44.836394 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-5175c552-4bb3-45da-8b7e-b9d13e3b494e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:44.879486 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 876/1754] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:44.896177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175c552-4bb3-45da-8b7e-b9d13e3b494e None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:44.914561 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175c552-4bb3-45da-8b7e-b9d13e3b494e None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:44.920254 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c46a3-84d7-4dbe-8e44-f4bda535c230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:44.921179 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c46a3-84d7-4dbe-8e44-f4bda535c230 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:44.922366 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c46a3-84d7-4dbe-8e44-f4bda535c230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:44.923200 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c46a3-84d7-4dbe-8e44-f4bda535c230 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:44.924341 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c46a3-84d7-4dbe-8e44-f4bda535c230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:44.926917 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c46a3-84d7-4dbe-8e44-f4bda535c230 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:44.946898 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 877/1755] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:44.946898 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c46a3-84d7-4dbe-8e44-f4bda535c230 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:44.946898 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-451c46a3-84d7-4dbe-8e44-f4bda535c230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:45.093862 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c46a3-84d7-4dbe-8e44-f4bda535c230 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:45.115549 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d28af-9ebc-4199-b5ab-aa85f75900a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:45.116395 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d28af-9ebc-4199-b5ab-aa85f75900a1 service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:45.116758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d28af-9ebc-4199-b5ab-aa85f75900a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:45.117135 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d28af-9ebc-4199-b5ab-aa85f75900a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:45.126325 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:45.129689 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d28af-9ebc-4199-b5ab-aa85f75900a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:45.133458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d28af-9ebc-4199-b5ab-aa85f75900a1 service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:45.139238 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 879/1756] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:44 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:45.145951 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c59a1-266c-4aa8-a122-78488298562e None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:45.155567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 878/1757] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:44 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:45.156548 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d28af-9ebc-4199-b5ab-aa85f75900a1 service nova] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:45.158177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9f2d28af-9ebc-4199-b5ab-aa85f75900a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:45.173287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d28af-9ebc-4199-b5ab-aa85f75900a1 service nova] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:45.179215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e249a0-29a2-422c-8ec5-6097d540cca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:45.180001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e249a0-29a2-422c-8ec5-6097d540cca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:45.180381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e249a0-29a2-422c-8ec5-6097d540cca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:45.180993 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e249a0-29a2-422c-8ec5-6097d540cca2 admin admin] PATH_INFO: `/v3/projects/7dfda4531b664d18afd50da65858bacb` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:45.182102 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e249a0-29a2-422c-8ec5-6097d540cca2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dfda4531b664d18afd50da65858bacb)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:45.184499 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e249a0-29a2-422c-8ec5-6097d540cca2 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:45.218741 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c59a1-266c-4aa8-a122-78488298562e None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:45.228696 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8a926-8960-42a0-bb3d-8f280508e74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:45.229800 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8a926-8960-42a0-bb3d-8f280508e74a admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:45.230449 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8a926-8960-42a0-bb3d-8f280508e74a admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:45.230941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8a926-8960-42a0-bb3d-8f280508e74a admin admin] PATH_INFO: `/v3/projects/a167c5de72034f2e8a657ef35e82da08/users/a5f52c113713454c92757b32f76bb564/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:45.274182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c8a926-8960-42a0-bb3d-8f280508e74a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a167c5de72034f2e8a657ef35e82da08, user_id=a5f52c113713454c92757b32f76bb564, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:45.283352 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c8a926-8960-42a0-bb3d-8f280508e74a admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:45.291403 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-07e249a0-29a2-422c-8ec5-6097d540cca2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dfda4531b664d18afd50da65858bacb', 'request_id': 'req-07e249a0-29a2-422c-8ec5-6097d540cca2'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:45.307832 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-07e249a0-29a2-422c-8ec5-6097d540cca2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dfda4531b664d18afd50da65858bacb', 'request_id': 'req-07e249a0-29a2-422c-8ec5-6097d540cca2'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:45.314935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 880/1758] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:45 2022] PUT /identity/v3/projects/a167c5de72034f2e8a657ef35e82da08/users/a5f52c113713454c92757b32f76bb564/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:45.325470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8a926-8960-42a0-bb3d-8f280508e74a admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:45.327289 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-07e249a0-29a2-422c-8ec5-6097d540cca2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dfda4531b664d18afd50da65858bacb', 'request_id': 'req-07e249a0-29a2-422c-8ec5-6097d540cca2'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:45.349448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-07e249a0-29a2-422c-8ec5-6097d540cca2 admin admin] The token cache is being invalidate because project 7dfda4531b664d18afd50da65858bacb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:12:45.349799 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-07e249a0-29a2-422c-8ec5-6097d540cca2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:45.350451 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-07e249a0-29a2-422c-8ec5-6097d540cca2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:45.352015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 879/1759] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:12:45 2022] DELETE /identity/v3/projects/7dfda4531b664d18afd50da65858bacb => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:45.368700 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a8450-0131-4c0d-b6a9-0876fe5494d0 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:45.368993 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a8450-0131-4c0d-b6a9-0876fe5494d0 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:45.369272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a8450-0131-4c0d-b6a9-0876fe5494d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:45.431712 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-dd6a8450-0131-4c0d-b6a9-0876fe5494d0 None None] MFA Rules not processed for user `a5f52c113713454c92757b32f76bb564`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:45.432954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-dd6a8450-0131-4c0d-b6a9-0876fe5494d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:45.491645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8a926-8960-42a0-bb3d-8f280508e74a admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:45.505298 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504901a-f59d-4a44-95a3-e117d0fefb4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:45.505958 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504901a-f59d-4a44-95a3-e117d0fefb4c service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:45.506234 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504901a-f59d-4a44-95a3-e117d0fefb4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:45.506508 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504901a-f59d-4a44-95a3-e117d0fefb4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:45.512165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f504901a-f59d-4a44-95a3-e117d0fefb4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:45.523683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:45.667790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f504901a-f59d-4a44-95a3-e117d0fefb4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:45.669290 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f504901a-f59d-4a44-95a3-e117d0fefb4c service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:45.680951 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 880/1760] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:45.689141 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a8450-0131-4c0d-b6a9-0876fe5494d0 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:45.705784 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-dd6a8450-0131-4c0d-b6a9-0876fe5494d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:45.708758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 881/1761] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:45 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:45.710317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504901a-f59d-4a44-95a3-e117d0fefb4c service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:45.716777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f504901a-f59d-4a44-95a3-e117d0fefb4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:45.931378 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a8450-0131-4c0d-b6a9-0876fe5494d0 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:45.937586 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c396c6e6-2f57-42ea-86a8-0b3aa345c650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:45.949578 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396c6e6-2f57-42ea-86a8-0b3aa345c650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:45.949891 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396c6e6-2f57-42ea-86a8-0b3aa345c650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:45.950172 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396c6e6-2f57-42ea-86a8-0b3aa345c650 admin admin] PATH_INFO: `/v3/users/cd489aa3dded4c149d10de2801798e27` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:45.955303 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c396c6e6-2f57-42ea-86a8-0b3aa345c650 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd489aa3dded4c149d10de2801798e27)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:45.956730 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504901a-f59d-4a44-95a3-e117d0fefb4c service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:45.958079 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c396c6e6-2f57-42ea-86a8-0b3aa345c650 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:45.966088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c663e8-afb7-4a0a-85e2-2c07462a1f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:45.967407 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c663e8-afb7-4a0a-85e2-2c07462a1f76 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:45.967706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:45.968996 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c663e8-afb7-4a0a-85e2-2c07462a1f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:45.969528 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c663e8-afb7-4a0a-85e2-2c07462a1f76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:45.972006 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c663e8-afb7-4a0a-85e2-2c07462a1f76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:45.976524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c663e8-afb7-4a0a-85e2-2c07462a1f76 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:46.063071 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c396c6e6-2f57-42ea-86a8-0b3aa345c650 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd489aa3dded4c149d10de2801798e27'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:46.072470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 882/1762] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:45 2022] POST /identity/v3/projects => generated 393 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:46.075062 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c396c6e6-2f57-42ea-86a8-0b3aa345c650 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd489aa3dded4c149d10de2801798e27'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:46.077147 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c663e8-afb7-4a0a-85e2-2c07462a1f76 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:46.090530 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-68c663e8-afb7-4a0a-85e2-2c07462a1f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:46.124082 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c396c6e6-2f57-42ea-86a8-0b3aa345c650 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd489aa3dded4c149d10de2801798e27'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:46.158600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c396c6e6-2f57-42ea-86a8-0b3aa345c650 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd489aa3dded4c149d10de2801798e27'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:46.160104 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c396c6e6-2f57-42ea-86a8-0b3aa345c650 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd489aa3dded4c149d10de2801798e27'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:46.175635 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 881/1763] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:12:45 2022] DELETE /identity/v3/users/cd489aa3dded4c149d10de2801798e27 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:46.175635 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c396c6e6-2f57-42ea-86a8-0b3aa345c650 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:46.178071 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c396c6e6-2f57-42ea-86a8-0b3aa345c650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:46.276063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c663e8-afb7-4a0a-85e2-2c07462a1f76 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:46.341430 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c396c6e6-2f57-42ea-86a8-0b3aa345c650 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:46.349427 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:46.358186 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe68be2-251a-4d05-b4a1-1aaa2ccb5320 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:46.360027 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe68be2-251a-4d05-b4a1-1aaa2ccb5320 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:46.361554 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe68be2-251a-4d05-b4a1-1aaa2ccb5320 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:46.363399 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe68be2-251a-4d05-b4a1-1aaa2ccb5320 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:46.369109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-fbe68be2-251a-4d05-b4a1-1aaa2ccb5320 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:46.406247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edb3a5-bc12-49e4-aab7-3c4b43208444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:46.406841 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edb3a5-bc12-49e4-aab7-3c4b43208444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:46.407512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edb3a5-bc12-49e4-aab7-3c4b43208444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:46.407793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edb3a5-bc12-49e4-aab7-3c4b43208444 admin admin] PATH_INFO: `/v3/projects/445d138b2f984ffebd1dd3aef0b97ff5` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:46.409246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edb3a5-bc12-49e4-aab7-3c4b43208444 admin admin] RBAC: Authorizing `identity:delete_project(project_id=445d138b2f984ffebd1dd3aef0b97ff5)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:46.412318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edb3a5-bc12-49e4-aab7-3c4b43208444 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:46.540852 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-52edb3a5-bc12-49e4-aab7-3c4b43208444 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '445d138b2f984ffebd1dd3aef0b97ff5', 'request_id': 'req-52edb3a5-bc12-49e4-aab7-3c4b43208444'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:46.553514 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-52edb3a5-bc12-49e4-aab7-3c4b43208444 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '445d138b2f984ffebd1dd3aef0b97ff5', 'request_id': 'req-52edb3a5-bc12-49e4-aab7-3c4b43208444'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:46.577011 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-52edb3a5-bc12-49e4-aab7-3c4b43208444 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '445d138b2f984ffebd1dd3aef0b97ff5', 'request_id': 'req-52edb3a5-bc12-49e4-aab7-3c4b43208444'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:46.594610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-52edb3a5-bc12-49e4-aab7-3c4b43208444 admin admin] The token cache is being invalidate because project 445d138b2f984ffebd1dd3aef0b97ff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:12:46.594947 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-52edb3a5-bc12-49e4-aab7-3c4b43208444 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:46.598563 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-52edb3a5-bc12-49e4-aab7-3c4b43208444 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:46.600561 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 883/1764] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:12:46 2022] DELETE /identity/v3/projects/445d138b2f984ffebd1dd3aef0b97ff5 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:46.608190 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edb3a5-bc12-49e4-aab7-3c4b43208444 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:46.616258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-52edb3a5-bc12-49e4-aab7-3c4b43208444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:46.627259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe68be2-251a-4d05-b4a1-1aaa2ccb5320 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:46.628771 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe68be2-251a-4d05-b4a1-1aaa2ccb5320 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:46.667796 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 882/1765] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:46.668941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302fff9-8b49-42a9-b23c-a48d57de86b2 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:46.669814 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302fff9-8b49-42a9-b23c-a48d57de86b2 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:46.670107 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302fff9-8b49-42a9-b23c-a48d57de86b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:46.761270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-9302fff9-8b49-42a9-b23c-a48d57de86b2 None None] MFA Rules not processed for user `2420a84d76e845c68bd14852b1a910a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:46.762653 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9302fff9-8b49-42a9-b23c-a48d57de86b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:46.845373 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:46.857744 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edb3a5-bc12-49e4-aab7-3c4b43208444 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:46.878780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb3777-9597-413e-8ed7-a85bc1e01bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:46.880680 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb3777-9597-413e-8ed7-a85bc1e01bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:46.881012 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb3777-9597-413e-8ed7-a85bc1e01bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:46.882716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb3777-9597-413e-8ed7-a85bc1e01bc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:46.883397 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eb3777-9597-413e-8ed7-a85bc1e01bc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:46.895688 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eb3777-9597-413e-8ed7-a85bc1e01bc6 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:46.943818 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 883/1766] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:46.945497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d3710-e39d-4029-9f85-89b19ecf6c14 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:46.945623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d3710-e39d-4029-9f85-89b19ecf6c14 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:46.945938 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d3710-e39d-4029-9f85-89b19ecf6c14 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:46.952849 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 884/1767] 10.222.0.73 () {54 vars in 912 bytes} [Mon Aug 29 20:12:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:12:46.954181 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d3710-e39d-4029-9f85-89b19ecf6c14 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:46.957055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-4c6d3710-e39d-4029-9f85-89b19ecf6c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:47.007250 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 884/1768] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:46 2022] POST /identity/v3/users => generated 303 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:47.009174 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61ce37-40b8-44ed-8919-88ebcd31c933 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:47.009859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61ce37-40b8-44ed-8919-88ebcd31c933 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:47.011656 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61ce37-40b8-44ed-8919-88ebcd31c933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:47.016144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d3710-e39d-4029-9f85-89b19ecf6c14 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:47.038360 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe22f6-1a65-4e57-9ede-3b7f810a19fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:47.039225 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe22f6-1a65-4e57-9ede-3b7f810a19fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:47.039680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe22f6-1a65-4e57-9ede-3b7f810a19fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:47.040385 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe22f6-1a65-4e57-9ede-3b7f810a19fd admin admin] PATH_INFO: `/v3/users/746ad1d9ffdf4b3084e9a2044bf2aa48` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:47.043908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe22f6-1a65-4e57-9ede-3b7f810a19fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=746ad1d9ffdf4b3084e9a2044bf2aa48)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:47.049872 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe22f6-1a65-4e57-9ede-3b7f810a19fd admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:47.099305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-2a61ce37-40b8-44ed-8919-88ebcd31c933 None None] MFA Rules not processed for user `2420a84d76e845c68bd14852b1a910a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:47.104050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-2a61ce37-40b8-44ed-8919-88ebcd31c933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:47.129940 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 885/1769] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:47.131274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39ff17-d10b-4b19-8554-d68711147ea6 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:47.131780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39ff17-d10b-4b19-8554-d68711147ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:47.131886 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39ff17-d10b-4b19-8554-d68711147ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:47.270079 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-ff39ff17-d10b-4b19-8554-d68711147ea6 None None] MFA Rules not processed for user `64ba58d774674ed7b233ad6416db068a`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:47.273257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-89fe22f6-1a65-4e57-9ede-3b7f810a19fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '746ad1d9ffdf4b3084e9a2044bf2aa48'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:47.277459 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ff39ff17-d10b-4b19-8554-d68711147ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:47.286467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-89fe22f6-1a65-4e57-9ede-3b7f810a19fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '746ad1d9ffdf4b3084e9a2044bf2aa48'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:47.333988 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-89fe22f6-1a65-4e57-9ede-3b7f810a19fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '746ad1d9ffdf4b3084e9a2044bf2aa48'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:47.341605 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:47.355264 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-89fe22f6-1a65-4e57-9ede-3b7f810a19fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '746ad1d9ffdf4b3084e9a2044bf2aa48'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:47.356044 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-89fe22f6-1a65-4e57-9ede-3b7f810a19fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '746ad1d9ffdf4b3084e9a2044bf2aa48'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:47.371612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 886/1771] 10.222.0.73 () {58 vars in 1009 bytes} [Mon Aug 29 20:12:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:47.373089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39ff17-d10b-4b19-8554-d68711147ea6 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:47.376334 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ff39ff17-d10b-4b19-8554-d68711147ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:47.378681 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 885/1771] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:12:46 2022] DELETE /identity/v3/users/746ad1d9ffdf4b3084e9a2044bf2aa48 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:47.379845 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe22f6-1a65-4e57-9ede-3b7f810a19fd admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:47.384412 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-89fe22f6-1a65-4e57-9ede-3b7f810a19fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:47.416413 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39ff17-d10b-4b19-8554-d68711147ea6 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:47.489555 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4a639-9ac4-4074-8589-a277889c47db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:47.490211 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4a639-9ac4-4074-8589-a277889c47db admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:47.490609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4a639-9ac4-4074-8589-a277889c47db admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:47.490910 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4a639-9ac4-4074-8589-a277889c47db admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:47.491936 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4a639-9ac4-4074-8589-a277889c47db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:47.493612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4a639-9ac4-4074-8589-a277889c47db admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:47.511964 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 887/1772] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:47.513179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4a639-9ac4-4074-8589-a277889c47db admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:47.515189 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-d3e4a639-9ac4-4074-8589-a277889c47db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:47.529776 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4a639-9ac4-4074-8589-a277889c47db admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:47.535217 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad45f6d-8d9d-4267-aceb-5b18d526951d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:47.535956 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad45f6d-8d9d-4267-aceb-5b18d526951d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:47.536313 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad45f6d-8d9d-4267-aceb-5b18d526951d admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:47.536665 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad45f6d-8d9d-4267-aceb-5b18d526951d admin admin] PATH_INFO: `/v3/projects/dba3bce9a4e74b8e8f5be51b15df720d` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:47.538330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad45f6d-8d9d-4267-aceb-5b18d526951d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dba3bce9a4e74b8e8f5be51b15df720d)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:47.540075 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad45f6d-8d9d-4267-aceb-5b18d526951d admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:47.599901 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe22f6-1a65-4e57-9ede-3b7f810a19fd admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:47.604285 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-4ad45f6d-8d9d-4267-aceb-5b18d526951d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dba3bce9a4e74b8e8f5be51b15df720d', 'request_id': 'req-4ad45f6d-8d9d-4267-aceb-5b18d526951d'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:47.616444 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-4ad45f6d-8d9d-4267-aceb-5b18d526951d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dba3bce9a4e74b8e8f5be51b15df720d', 'request_id': 'req-4ad45f6d-8d9d-4267-aceb-5b18d526951d'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:47.626573 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-4ad45f6d-8d9d-4267-aceb-5b18d526951d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dba3bce9a4e74b8e8f5be51b15df720d', 'request_id': 'req-4ad45f6d-8d9d-4267-aceb-5b18d526951d'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:47.642455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-4ad45f6d-8d9d-4267-aceb-5b18d526951d admin admin] The token cache is being invalidate because project dba3bce9a4e74b8e8f5be51b15df720d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:12:47.643157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-4ad45f6d-8d9d-4267-aceb-5b18d526951d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:47.644310 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-4ad45f6d-8d9d-4267-aceb-5b18d526951d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:47.651568 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 888/1773] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:12:47 2022] DELETE /identity/v3/projects/dba3bce9a4e74b8e8f5be51b15df720d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:47.653551 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad45f6d-8d9d-4267-aceb-5b18d526951d admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:47.657665 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-4ad45f6d-8d9d-4267-aceb-5b18d526951d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:47.688068 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:47.810483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad45f6d-8d9d-4267-aceb-5b18d526951d admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:47.811988 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd92f0-417b-4aeb-8835-c7e77b82bb3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:47.814735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd92f0-417b-4aeb-8835-c7e77b82bb3f service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:47.816397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd92f0-417b-4aeb-8835-c7e77b82bb3f service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:47.816397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd92f0-417b-4aeb-8835-c7e77b82bb3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:47.822556 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-35dd92f0-417b-4aeb-8835-c7e77b82bb3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:47.824444 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2792c8-2ea2-40ba-9175-49eaace9f8a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:47.825088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2792c8-2ea2-40ba-9175-49eaace9f8a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:47.825374 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2792c8-2ea2-40ba-9175-49eaace9f8a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:47.826257 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2792c8-2ea2-40ba-9175-49eaace9f8a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:47.843255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ca2792c8-2ea2-40ba-9175-49eaace9f8a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:47.877786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2792c8-2ea2-40ba-9175-49eaace9f8a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:47.879261 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2792c8-2ea2-40ba-9175-49eaace9f8a9 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:47.919156 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 889/1774] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:47.921509 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2792c8-2ea2-40ba-9175-49eaace9f8a9 service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:47.930878 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ca2792c8-2ea2-40ba-9175-49eaace9f8a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:47.931344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd92f0-417b-4aeb-8835-c7e77b82bb3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:47.933308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd92f0-417b-4aeb-8835-c7e77b82bb3f service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:47.950215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:47.969209 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 886/1775] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:47.977008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e7fc2-93f7-4ba1-92cb-a99fc4103a29 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:47.977008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e7fc2-93f7-4ba1-92cb-a99fc4103a29 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:47.977008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e7fc2-93f7-4ba1-92cb-a99fc4103a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:48.009890 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-642e7fc2-93f7-4ba1-92cb-a99fc4103a29 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:48.011618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-642e7fc2-93f7-4ba1-92cb-a99fc4103a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:48.153093 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2792c8-2ea2-40ba-9175-49eaace9f8a9 service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:48.163889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480299c-b228-4926-9182-71eae98d3611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:48.164874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480299c-b228-4926-9182-71eae98d3611 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:48.165299 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480299c-b228-4926-9182-71eae98d3611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:48.165762 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480299c-b228-4926-9182-71eae98d3611 admin admin] PATH_INFO: `/v3/projects/bf833eda0ba94fbeaf7f2c60ae84fb0d/users/df4115d2479046e1b3f2a328ee7828cf/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:48.198674 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480299c-b228-4926-9182-71eae98d3611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf833eda0ba94fbeaf7f2c60ae84fb0d, user_id=df4115d2479046e1b3f2a328ee7828cf, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:48.202281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480299c-b228-4926-9182-71eae98d3611 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:48.234132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 887/1776] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:48.241250 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72315976-8398-4fe1-ad78-eb610fd2c02c None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:48.242272 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 890/1777] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:47 2022] PUT /identity/v3/projects/bf833eda0ba94fbeaf7f2c60ae84fb0d/users/df4115d2479046e1b3f2a328ee7828cf/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:48.242422 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72315976-8398-4fe1-ad78-eb610fd2c02c None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:48.242680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72315976-8398-4fe1-ad78-eb610fd2c02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:48.244989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e125ee-80b8-4033-965b-752b42693a65 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:48.246360 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e125ee-80b8-4033-965b-752b42693a65 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:48.246650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e125ee-80b8-4033-965b-752b42693a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:48.296227 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-b3e125ee-80b8-4033-965b-752b42693a65 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:48.299720 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-b3e125ee-80b8-4033-965b-752b42693a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:48.314313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-72315976-8398-4fe1-ad78-eb610fd2c02c None None] MFA Rules not processed for user `04c2aae2da284df29243b99d33725ddd`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:48.315989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-72315976-8398-4fe1-ad78-eb610fd2c02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:48.443939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 888/1778] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:48.444816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72315976-8398-4fe1-ad78-eb610fd2c02c None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:48.452221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 891/1779] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:48.456097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e125ee-80b8-4033-965b-752b42693a65 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:48.465492 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72315976-8398-4fe1-ad78-eb610fd2c02c None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:48.479116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e125ee-80b8-4033-965b-752b42693a65 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:48.493864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05e773-2a01-4e3b-b85b-c992b7c084ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:48.494016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ffa84-ca1a-4794-bcd0-4f3c2cdb4c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:48.494244 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05e773-2a01-4e3b-b85b-c992b7c084ac admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:48.494350 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ffa84-ca1a-4794-bcd0-4f3c2cdb4c00 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:48.494526 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05e773-2a01-4e3b-b85b-c992b7c084ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:48.494626 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ffa84-ca1a-4794-bcd0-4f3c2cdb4c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:48.494805 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05e773-2a01-4e3b-b85b-c992b7c084ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:48.494901 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ffa84-ca1a-4794-bcd0-4f3c2cdb4c00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:48.495525 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc05e773-2a01-4e3b-b85b-c992b7c084ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:12:48.495615 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ffa84-ca1a-4794-bcd0-4f3c2cdb4c00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:48.495721 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc05e773-2a01-4e3b-b85b-c992b7c084ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:48.503449 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc05e773-2a01-4e3b-b85b-c992b7c084ac admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:48.504139 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ffa84-ca1a-4794-bcd0-4f3c2cdb4c00 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:48.542585 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 892/1780] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:12:48.542585 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05e773-2a01-4e3b-b85b-c992b7c084ac admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:48.548939 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-fc05e773-2a01-4e3b-b85b-c992b7c084ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:48.560943 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 889/1781] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:48.563367 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ffa84-ca1a-4794-bcd0-4f3c2cdb4c00 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:48.596857 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ffa84-ca1a-4794-bcd0-4f3c2cdb4c00 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:48.613884 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df51c4-1803-494b-9c71-3dde5c946793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:48.614725 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df51c4-1803-494b-9c71-3dde5c946793 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:48.615180 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df51c4-1803-494b-9c71-3dde5c946793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:48.615626 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df51c4-1803-494b-9c71-3dde5c946793 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:48.616591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df51c4-1803-494b-9c71-3dde5c946793 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:48.618711 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05e773-2a01-4e3b-b85b-c992b7c084ac admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:48.624260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df51c4-1803-494b-9c71-3dde5c946793 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:48.639335 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15320de9-71c9-4036-b969-f7bc7ca19ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:48.640002 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15320de9-71c9-4036-b969-f7bc7ca19ca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:48.640282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15320de9-71c9-4036-b969-f7bc7ca19ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:48.640554 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15320de9-71c9-4036-b969-f7bc7ca19ca2 admin admin] PATH_INFO: `/v3/projects/6f4cd662f318463d8d8f0ea7a053b22e` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:48.646989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:48.648800 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15320de9-71c9-4036-b969-f7bc7ca19ca2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f4cd662f318463d8d8f0ea7a053b22e)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:48.651413 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15320de9-71c9-4036-b969-f7bc7ca19ca2 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:48.692325 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 890/1782] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:48 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:48.693184 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df51c4-1803-494b-9c71-3dde5c946793 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:48.793670 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-15320de9-71c9-4036-b969-f7bc7ca19ca2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f4cd662f318463d8d8f0ea7a053b22e', 'request_id': 'req-15320de9-71c9-4036-b969-f7bc7ca19ca2'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:48.802418 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-15320de9-71c9-4036-b969-f7bc7ca19ca2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f4cd662f318463d8d8f0ea7a053b22e', 'request_id': 'req-15320de9-71c9-4036-b969-f7bc7ca19ca2'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:48.813822 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-15320de9-71c9-4036-b969-f7bc7ca19ca2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f4cd662f318463d8d8f0ea7a053b22e', 'request_id': 'req-15320de9-71c9-4036-b969-f7bc7ca19ca2'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:48.827007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:48.837454 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-15320de9-71c9-4036-b969-f7bc7ca19ca2 admin admin] The token cache is being invalidate because project 6f4cd662f318463d8d8f0ea7a053b22e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:12:48.838199 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-15320de9-71c9-4036-b969-f7bc7ca19ca2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:48.840650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-15320de9-71c9-4036-b969-f7bc7ca19ca2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:48.847742 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df51c4-1803-494b-9c71-3dde5c946793 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:48.861999 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 893/1783] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:12:48 2022] DELETE /identity/v3/projects/6f4cd662f318463d8d8f0ea7a053b22e => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:48.863477 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15320de9-71c9-4036-b969-f7bc7ca19ca2 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:48.869589 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-15320de9-71c9-4036-b969-f7bc7ca19ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:49.000539 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a36fe1-b041-4edb-8b13-5dc1f8b20a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:49.001242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a36fe1-b041-4edb-8b13-5dc1f8b20a22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:49.001521 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a36fe1-b041-4edb-8b13-5dc1f8b20a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:49.001819 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a36fe1-b041-4edb-8b13-5dc1f8b20a22 admin admin] PATH_INFO: `/v3/projects/bf833eda0ba94fbeaf7f2c60ae84fb0d/users/df4115d2479046e1b3f2a328ee7828cf/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:49.003855 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a36fe1-b041-4edb-8b13-5dc1f8b20a22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf833eda0ba94fbeaf7f2c60ae84fb0d, user_id=df4115d2479046e1b3f2a328ee7828cf, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:49.010446 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a36fe1-b041-4edb-8b13-5dc1f8b20a22 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:49.049329 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 891/1784] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:48 2022] PUT /identity/v3/projects/bf833eda0ba94fbeaf7f2c60ae84fb0d/users/df4115d2479046e1b3f2a328ee7828cf/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:49.050076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15320de9-71c9-4036-b969-f7bc7ca19ca2 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:49.051187 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244a6ea-c78e-4f7e-819f-6c1604d334a7 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:49.051187 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244a6ea-c78e-4f7e-819f-6c1604d334a7 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:49.051187 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244a6ea-c78e-4f7e-819f-6c1604d334a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:49.077814 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-6244a6ea-c78e-4f7e-819f-6c1604d334a7 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:49.079499 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-6244a6ea-c78e-4f7e-819f-6c1604d334a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:49.149665 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd648c2-bba1-416a-8050-6be0225ab576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:49.150391 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd648c2-bba1-416a-8050-6be0225ab576 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:49.150681 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd648c2-bba1-416a-8050-6be0225ab576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:49.150962 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd648c2-bba1-416a-8050-6be0225ab576 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:49.151601 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd648c2-bba1-416a-8050-6be0225ab576 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:49.154526 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd648c2-bba1-416a-8050-6be0225ab576 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:49.181212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 894/1785] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:48 2022] POST /identity/v3/users => generated 287 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:49.183276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92ed4a-1b86-4b9b-a19c-b5d1691828cf None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:49.184599 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92ed4a-1b86-4b9b-a19c-b5d1691828cf None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:49.186503 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92ed4a-1b86-4b9b-a19c-b5d1691828cf None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:49.188583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 895/1786] 10.222.0.73 () {54 vars in 912 bytes} [Mon Aug 29 20:12:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:12:49.190370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc04dd-7559-40c2-9f7a-82486177748c None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:49.190866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc04dd-7559-40c2-9f7a-82486177748c None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:49.191303 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc04dd-7559-40c2-9f7a-82486177748c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:49.236333 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 892/1787] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:49.237004 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244a6ea-c78e-4f7e-819f-6c1604d334a7 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:49.250586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-94cc04dd-7559-40c2-9f7a-82486177748c None None] MFA Rules not processed for user `df4115d2479046e1b3f2a328ee7828cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:49.252699 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-94cc04dd-7559-40c2-9f7a-82486177748c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:49.268133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244a6ea-c78e-4f7e-819f-6c1604d334a7 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:49.292895 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c7b59-ba7b-4fbb-8b07-44fdce103f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:49.295526 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c7b59-ba7b-4fbb-8b07-44fdce103f7d admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:49.298668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c7b59-ba7b-4fbb-8b07-44fdce103f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:49.299737 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c7b59-ba7b-4fbb-8b07-44fdce103f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:49.301383 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c7b59-ba7b-4fbb-8b07-44fdce103f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:49.306312 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c7b59-ba7b-4fbb-8b07-44fdce103f7d admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:49.333896 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 893/1788] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:49 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:49.335679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37f7fd-b33c-4026-b9b3-04985aeb7526 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:49.336314 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37f7fd-b33c-4026-b9b3-04985aeb7526 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:49.336787 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37f7fd-b33c-4026-b9b3-04985aeb7526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:49.360937 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 896/1789] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:49 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:49.362119 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc04dd-7559-40c2-9f7a-82486177748c None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:49.365121 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-94cc04dd-7559-40c2-9f7a-82486177748c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:49.383657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-ac37f7fd-b33c-4026-b9b3-04985aeb7526 None None] MFA Rules not processed for user `64ba58d774674ed7b233ad6416db068a`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:49.385116 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-ac37f7fd-b33c-4026-b9b3-04985aeb7526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:49.394932 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:49.403175 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:49.495810 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc04dd-7559-40c2-9f7a-82486177748c None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:49.500128 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 894/1790] 10.222.0.73 () {58 vars in 1009 bytes} [Mon Aug 29 20:12:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:49.508149 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658bb6b-1960-45a7-9ebd-b3e1b95ad081 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:49.508589 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658bb6b-1960-45a7-9ebd-b3e1b95ad081 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:49.508879 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658bb6b-1960-45a7-9ebd-b3e1b95ad081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:49.526490 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f4349-07cd-4b5f-b4a2-f5f76b05118a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab2e318b2854c46bc7ed2151a44a2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:49.529351 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f4349-07cd-4b5f-b4a2-f5f76b05118a service placement] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:49.529837 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f4349-07cd-4b5f-b4a2-f5f76b05118a service placement] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:49.532192 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f4349-07cd-4b5f-b4a2-f5f76b05118a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:49.534989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-3658bb6b-1960-45a7-9ebd-b3e1b95ad081 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:49.536213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-7c8f4349-07cd-4b5f-b4a2-f5f76b05118a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:49.536779 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-3658bb6b-1960-45a7-9ebd-b3e1b95ad081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:49.547398 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 895/1791] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:49.548715 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658bb6b-1960-45a7-9ebd-b3e1b95ad081 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:49.564243 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8f4349-07cd-4b5f-b4a2-f5f76b05118a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:49.567467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658bb6b-1960-45a7-9ebd-b3e1b95ad081 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:49.585983 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c319f04-6881-42b5-8bc3-78be7f999e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:49.586717 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8f4349-07cd-4b5f-b4a2-f5f76b05118a service placement] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:49.586848 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c319f04-6881-42b5-8bc3-78be7f999e0c admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:49.587010 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c319f04-6881-42b5-8bc3-78be7f999e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:49.587284 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c319f04-6881-42b5-8bc3-78be7f999e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:49.587953 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c319f04-6881-42b5-8bc3-78be7f999e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:49.589615 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c319f04-6881-42b5-8bc3-78be7f999e0c admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:49.612781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 896/1792] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:49 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:49.613774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c319f04-6881-42b5-8bc3-78be7f999e0c admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:49.618865 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 897/1793] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:49.619926 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f4349-07cd-4b5f-b4a2-f5f76b05118a service placement] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:49.620328 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-5c319f04-6881-42b5-8bc3-78be7f999e0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:49.625090 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-7c8f4349-07cd-4b5f-b4a2-f5f76b05118a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:49.679241 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f4349-07cd-4b5f-b4a2-f5f76b05118a service placement] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:49.693154 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4e9f8-4803-4b9f-9f42-6ad09937d800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:49.693809 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4e9f8-4803-4b9f-9f42-6ad09937d800 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:49.694142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4e9f8-4803-4b9f-9f42-6ad09937d800 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:49.694469 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4e9f8-4803-4b9f-9f42-6ad09937d800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:49.707006 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e4e9f8-4803-4b9f-9f42-6ad09937d800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:49.711138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e4e9f8-4803-4b9f-9f42-6ad09937d800 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:49.743429 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 898/1794] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:49.744382 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4e9f8-4803-4b9f-9f42-6ad09937d800 service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:49.786006 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4e9f8-4803-4b9f-9f42-6ad09937d800 service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:49.803436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f463d39-d97b-485c-a579-87f3698c7d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:49.807337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f463d39-d97b-485c-a579-87f3698c7d8c admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:49.807562 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c319f04-6881-42b5-8bc3-78be7f999e0c admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:49.807711 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f463d39-d97b-485c-a579-87f3698c7d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:49.807711 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f463d39-d97b-485c-a579-87f3698c7d8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:49.810207 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f463d39-d97b-485c-a579-87f3698c7d8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:12:49.810707 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f463d39-d97b-485c-a579-87f3698c7d8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:49.815470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3e293-b07a-4466-9ddf-d2fd76f52e39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:49.816137 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3e293-b07a-4466-9ddf-d2fd76f52e39 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:49.816639 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3e293-b07a-4466-9ddf-d2fd76f52e39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:49.816639 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3e293-b07a-4466-9ddf-d2fd76f52e39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:49.816900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f463d39-d97b-485c-a579-87f3698c7d8c admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:49.849854 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3e293-b07a-4466-9ddf-d2fd76f52e39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:49.851321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3e293-b07a-4466-9ddf-d2fd76f52e39 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:49.859011 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 899/1795] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:12:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:12:49.860961 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f463d39-d97b-485c-a579-87f3698c7d8c admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:49.885905 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 897/1796] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:49 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:49.886863 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3e293-b07a-4466-9ddf-d2fd76f52e39 service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:49.910763 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f463d39-d97b-485c-a579-87f3698c7d8c admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:49.920751 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3e293-b07a-4466-9ddf-d2fd76f52e39 service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:49.926003 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887cf1f1-a95a-4b95-b23e-54bb8b87c4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:49.926694 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cf1f1-a95a-4b95-b23e-54bb8b87c4e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:49.926966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cf1f1-a95a-4b95-b23e-54bb8b87c4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:49.927246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cf1f1-a95a-4b95-b23e-54bb8b87c4e5 admin admin] PATH_INFO: `/v3/projects/c5163a9611df47069421fe45e55c7ab4/users/93dda2d3d8bd4d2ca4b80dc9a09fcf81/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:49.930425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a6085-05e0-4cef-b3cf-60c3a6cbb1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:49.931049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a6085-05e0-4cef-b3cf-60c3a6cbb1a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:49.931332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a6085-05e0-4cef-b3cf-60c3a6cbb1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:49.931607 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a6085-05e0-4cef-b3cf-60c3a6cbb1a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:49.932247 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a6085-05e0-4cef-b3cf-60c3a6cbb1a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:49.934377 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a6085-05e0-4cef-b3cf-60c3a6cbb1a3 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:49.962741 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887cf1f1-a95a-4b95-b23e-54bb8b87c4e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5163a9611df47069421fe45e55c7ab4, user_id=93dda2d3d8bd4d2ca4b80dc9a09fcf81, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:49.965780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887cf1f1-a95a-4b95-b23e-54bb8b87c4e5 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:49.994726 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 900/1797] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:49 2022] PUT /identity/v3/projects/c5163a9611df47069421fe45e55c7ab4/users/93dda2d3d8bd4d2ca4b80dc9a09fcf81/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:50.001482 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f741a42-80d9-4a44-96d9-8bf8f7c4d97c None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:50.001797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f741a42-80d9-4a44-96d9-8bf8f7c4d97c None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:50.002080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f741a42-80d9-4a44-96d9-8bf8f7c4d97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:50.012724 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 898/1798] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:49 2022] POST /identity/v3/projects => generated 401 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:50.014100 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a6085-05e0-4cef-b3cf-60c3a6cbb1a3 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:50.070412 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-2f741a42-80d9-4a44-96d9-8bf8f7c4d97c None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:50.075394 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-2f741a42-80d9-4a44-96d9-8bf8f7c4d97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:50.199268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 901/1799] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:12:50.201061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f741a42-80d9-4a44-96d9-8bf8f7c4d97c None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:50.202552 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-2f741a42-80d9-4a44-96d9-8bf8f7c4d97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:50.251486 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a6085-05e0-4cef-b3cf-60c3a6cbb1a3 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:50.270520 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357e910-8434-449e-b0d7-c99f2514ff65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:50.271154 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357e910-8434-449e-b0d7-c99f2514ff65 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:50.271441 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357e910-8434-449e-b0d7-c99f2514ff65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:50.271713 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357e910-8434-449e-b0d7-c99f2514ff65 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:50.272350 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357e910-8434-449e-b0d7-c99f2514ff65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:50.276424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357e910-8434-449e-b0d7-c99f2514ff65 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:50.343195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 899/1800] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:50 2022] POST /identity/v3/users => generated 301 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:50.347617 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f741a42-80d9-4a44-96d9-8bf8f7c4d97c None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:50.348148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357e910-8434-449e-b0d7-c99f2514ff65 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:50.374575 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9811da-260d-4dab-b248-679b9104e963 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:50.375358 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9811da-260d-4dab-b248-679b9104e963 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:50.375539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9811da-260d-4dab-b248-679b9104e963 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:50.375935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9811da-260d-4dab-b248-679b9104e963 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:50.379545 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-fd9811da-260d-4dab-b248-679b9104e963 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:50.403596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:50.456776 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:50.549112 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357e910-8434-449e-b0d7-c99f2514ff65 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:50.555008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10bb0a-d02d-4a3c-a67d-af7cba114f76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:50.555977 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10bb0a-d02d-4a3c-a67d-af7cba114f76 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:50.556270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10bb0a-d02d-4a3c-a67d-af7cba114f76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:50.556544 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10bb0a-d02d-4a3c-a67d-af7cba114f76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:50.575736 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9811da-260d-4dab-b248-679b9104e963 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:50.578246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9811da-260d-4dab-b248-679b9104e963 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:50.581140 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10bb0a-d02d-4a3c-a67d-af7cba114f76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:50.583379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10bb0a-d02d-4a3c-a67d-af7cba114f76 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:50.610573 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 900/1801] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:50.623975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10bb0a-d02d-4a3c-a67d-af7cba114f76 service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:50.625540 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 902/1802] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:50 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:50.688253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10bb0a-d02d-4a3c-a67d-af7cba114f76 service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:50.697441 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e17f7-34fa-4e56-870c-aefa4d63f3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:50.698107 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e17f7-34fa-4e56-870c-aefa4d63f3f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:50.698390 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e17f7-34fa-4e56-870c-aefa4d63f3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:50.700307 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e17f7-34fa-4e56-870c-aefa4d63f3f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:50.701060 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e17f7-34fa-4e56-870c-aefa4d63f3f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:50.702800 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e17f7-34fa-4e56-870c-aefa4d63f3f0 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:50.735692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 901/1803] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:50.739911 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9811da-260d-4dab-b248-679b9104e963 service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:50.780796 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9811da-260d-4dab-b248-679b9104e963 service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:50.815392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc68dce-f21d-4954-a159-844b3fb12475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:50.823899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc68dce-f21d-4954-a159-844b3fb12475 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:50.827389 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc68dce-f21d-4954-a159-844b3fb12475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:50.829784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc68dce-f21d-4954-a159-844b3fb12475 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:50.832392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc68dce-f21d-4954-a159-844b3fb12475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:50.835612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc68dce-f21d-4954-a159-844b3fb12475 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:50.875478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e17f7-34fa-4e56-870c-aefa4d63f3f0 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:50.891341 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 903/1804] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:50.936273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e17f7-34fa-4e56-870c-aefa4d63f3f0 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:50.993486 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf221b-2613-472e-badd-c227cf0eb7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:50.994592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf221b-2613-472e-badd-c227cf0eb7ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:50.995019 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf221b-2613-472e-badd-c227cf0eb7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:51.009116 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf221b-2613-472e-badd-c227cf0eb7ef admin admin] PATH_INFO: `/v3/projects/f4d9d8ab0f934a3d836e70edb7dbddf5/users/05e24d04aa604053b8dda62e67ec746b/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:51.029256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b3193-5219-483a-917d-5748c468ec4a None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:51.029589 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b3193-5219-483a-917d-5748c468ec4a None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:51.029940 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b3193-5219-483a-917d-5748c468ec4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:51.128907 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-5a3b3193-5219-483a-917d-5748c468ec4a None None] MFA Rules not processed for user `e75b537d44524de192282b10825293f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:51.130182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-5a3b3193-5219-483a-917d-5748c468ec4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:51.188715 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbf221b-2613-472e-badd-c227cf0eb7ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4d9d8ab0f934a3d836e70edb7dbddf5, user_id=05e24d04aa604053b8dda62e67ec746b, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:51.205737 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbf221b-2613-472e-badd-c227cf0eb7ef admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:51.350656 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 902/1805] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:50 2022] PUT /identity/v3/projects/f4d9d8ab0f934a3d836e70edb7dbddf5/users/05e24d04aa604053b8dda62e67ec746b/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:51.364206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769180e7-50c2-4dd3-acef-8616da3d04c8 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:51.364816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769180e7-50c2-4dd3-acef-8616da3d04c8 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:51.365182 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769180e7-50c2-4dd3-acef-8616da3d04c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:51.414251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 904/1806] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:12:51.480251 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-769180e7-50c2-4dd3-acef-8616da3d04c8 None None] MFA Rules not processed for user `05e24d04aa604053b8dda62e67ec746b`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:51.481513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-769180e7-50c2-4dd3-acef-8616da3d04c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:51.850501 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b3193-5219-483a-917d-5748c468ec4a None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:51.859883 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 903/1807] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:51 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 497 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:51.874547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b3193-5219-483a-917d-5748c468ec4a None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:51.904549 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769180e7-50c2-4dd3-acef-8616da3d04c8 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:51.906210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-769180e7-50c2-4dd3-acef-8616da3d04c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:52.231017 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9ce21-26a8-45d5-b70c-2616bbcc3259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:52.232847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9ce21-26a8-45d5-b70c-2616bbcc3259 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:52.233458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9ce21-26a8-45d5-b70c-2616bbcc3259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:52.233827 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9ce21-26a8-45d5-b70c-2616bbcc3259 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:52.234494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9ce21-26a8-45d5-b70c-2616bbcc3259 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:52.269164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9ce21-26a8-45d5-b70c-2616bbcc3259 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:52.313536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769180e7-50c2-4dd3-acef-8616da3d04c8 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:52.342884 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac4a1d-2691-482c-a367-6d9b518a5669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:52.343662 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac4a1d-2691-482c-a367-6d9b518a5669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:52.343972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac4a1d-2691-482c-a367-6d9b518a5669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:52.344274 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac4a1d-2691-482c-a367-6d9b518a5669 admin admin] PATH_INFO: `/v3/users/0de6bd593699495294a7def9a7b8be9b` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:52.349486 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac4a1d-2691-482c-a367-6d9b518a5669 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0de6bd593699495294a7def9a7b8be9b)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:52.351760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac4a1d-2691-482c-a367-6d9b518a5669 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:52.354076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 905/1808] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:51 2022] POST /identity/v3/projects => generated 401 bytes in 504 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:12:52.357610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9ce21-26a8-45d5-b70c-2616bbcc3259 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:52.411406 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:52.479668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b1ac4a1d-2691-482c-a367-6d9b518a5669 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0de6bd593699495294a7def9a7b8be9b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:52.514052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b1ac4a1d-2691-482c-a367-6d9b518a5669 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0de6bd593699495294a7def9a7b8be9b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:52.515897 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:52.541230 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b1ac4a1d-2691-482c-a367-6d9b518a5669 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0de6bd593699495294a7def9a7b8be9b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:52.565581 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b1ac4a1d-2691-482c-a367-6d9b518a5669 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0de6bd593699495294a7def9a7b8be9b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:52.568230 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b1ac4a1d-2691-482c-a367-6d9b518a5669 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0de6bd593699495294a7def9a7b8be9b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:52.582365 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 904/1809] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:12:51 2022] DELETE /identity/v3/users/0de6bd593699495294a7def9a7b8be9b => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:52.591161 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d793f4-1810-44d5-931c-1980f5ba0030 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:52.591969 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d793f4-1810-44d5-931c-1980f5ba0030 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:52.593022 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d793f4-1810-44d5-931c-1980f5ba0030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:52.651877 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-b3d793f4-1810-44d5-931c-1980f5ba0030 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:52.653971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-b3d793f4-1810-44d5-931c-1980f5ba0030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:52.661270 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9ce21-26a8-45d5-b70c-2616bbcc3259 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:52.774056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 905/1810] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:52.865440 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d55f4-bbb8-4f3e-90f3-d02f73ac16e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:52.866887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d55f4-bbb8-4f3e-90f3-d02f73ac16e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:52.867914 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d55f4-bbb8-4f3e-90f3-d02f73ac16e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:52.868358 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d55f4-bbb8-4f3e-90f3-d02f73ac16e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:52.872242 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d55f4-bbb8-4f3e-90f3-d02f73ac16e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:52.878410 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d793f4-1810-44d5-931c-1980f5ba0030 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:52.879471 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d55f4-bbb8-4f3e-90f3-d02f73ac16e5 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:52.881507 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-b3d793f4-1810-44d5-931c-1980f5ba0030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:52.960022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 906/1811] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:52 2022] POST /identity/v3/users => generated 307 bytes in 603 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:52.961080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d55f4-bbb8-4f3e-90f3-d02f73ac16e5 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:52.968353 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-0f8d55f4-bbb8-4f3e-90f3-d02f73ac16e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:53.028452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d55f4-bbb8-4f3e-90f3-d02f73ac16e5 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:53.059136 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70a41d-2e2b-4079-9f9a-be30099d7ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:53.064104 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70a41d-2e2b-4079-9f9a-be30099d7ceb admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:53.064333 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70a41d-2e2b-4079-9f9a-be30099d7ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:53.066088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70a41d-2e2b-4079-9f9a-be30099d7ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:53.071122 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70a41d-2e2b-4079-9f9a-be30099d7ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:53.072843 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70a41d-2e2b-4079-9f9a-be30099d7ceb admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:53.126682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 907/1812] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:53.131563 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70a41d-2e2b-4079-9f9a-be30099d7ceb admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:53.136510 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d793f4-1810-44d5-931c-1980f5ba0030 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:53.148607 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5b618-f2da-4555-aec6-a9e930a227a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:53.149796 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5b618-f2da-4555-aec6-a9e930a227a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:53.150323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5b618-f2da-4555-aec6-a9e930a227a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:53.150769 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5b618-f2da-4555-aec6-a9e930a227a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:53.178869 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5b618-f2da-4555-aec6-a9e930a227a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:53.179370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70a41d-2e2b-4079-9f9a-be30099d7ceb admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:53.180405 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5b618-f2da-4555-aec6-a9e930a227a1 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:53.187443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea3b2d-c908-4439-8dc8-875b82206d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:53.188957 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea3b2d-c908-4439-8dc8-875b82206d8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:53.189744 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea3b2d-c908-4439-8dc8-875b82206d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:53.190538 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea3b2d-c908-4439-8dc8-875b82206d8a admin admin] PATH_INFO: `/v3/projects/453aa10dbf2d4aeb81c81a8169a8bf3e/users/9861cfc554b54667854755a19c0be75a/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:53.210293 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 906/1813] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:53.211659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793d82c-6dec-4eff-b0c0-a589a7d4e841 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:53.211950 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793d82c-6dec-4eff-b0c0-a589a7d4e841 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:53.212227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793d82c-6dec-4eff-b0c0-a589a7d4e841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:53.237828 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea3b2d-c908-4439-8dc8-875b82206d8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=453aa10dbf2d4aeb81c81a8169a8bf3e, user_id=9861cfc554b54667854755a19c0be75a, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:53.241447 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea3b2d-c908-4439-8dc8-875b82206d8a admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:53.294747 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 908/1814] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:53 2022] PUT /identity/v3/projects/453aa10dbf2d4aeb81c81a8169a8bf3e/users/9861cfc554b54667854755a19c0be75a/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:53.296026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-d793d82c-6dec-4eff-b0c0-a589a7d4e841 None None] MFA Rules not processed for user `93dda2d3d8bd4d2ca4b80dc9a09fcf81`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:53.297969 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d793d82c-6dec-4eff-b0c0-a589a7d4e841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:53.303273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea3b2d-c908-4439-8dc8-875b82206d8a admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:53.472464 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:53.561021 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 907/1815] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:53.565438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793d82c-6dec-4eff-b0c0-a589a7d4e841 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:53.572241 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d793d82c-6dec-4eff-b0c0-a589a7d4e841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:53.621539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea3b2d-c908-4439-8dc8-875b82206d8a admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:53.630932 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793d82c-6dec-4eff-b0c0-a589a7d4e841 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:53.641571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e7667-e3ef-4137-91a1-6b882f6bf7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:53.644368 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e7667-e3ef-4137-91a1-6b882f6bf7e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:53.645155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e7667-e3ef-4137-91a1-6b882f6bf7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:53.645657 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e7667-e3ef-4137-91a1-6b882f6bf7e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:53.647127 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e7667-e3ef-4137-91a1-6b882f6bf7e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:53.648537 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32d3e9-2e41-4cca-891c-a49fd8523678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:53.649177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32d3e9-2e41-4cca-891c-a49fd8523678 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:53.649464 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32d3e9-2e41-4cca-891c-a49fd8523678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:53.649766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32d3e9-2e41-4cca-891c-a49fd8523678 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:53.653425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e7667-e3ef-4137-91a1-6b882f6bf7e7 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:53.655391 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32d3e9-2e41-4cca-891c-a49fd8523678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:53.657340 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32d3e9-2e41-4cca-891c-a49fd8523678 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:53.675977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 909/1816] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:53 2022] GET /identity/v3/roles => generated 1420 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:53.686424 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e7667-e3ef-4137-91a1-6b882f6bf7e7 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:53.733549 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e7667-e3ef-4137-91a1-6b882f6bf7e7 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:53.756976 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871e9fe-7d18-4a7a-b86f-698ab633e5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:53.758215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871e9fe-7d18-4a7a-b86f-698ab633e5e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:53.759039 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871e9fe-7d18-4a7a-b86f-698ab633e5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:53.761309 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871e9fe-7d18-4a7a-b86f-698ab633e5e5 admin admin] PATH_INFO: `/v3/projects/453aa10dbf2d4aeb81c81a8169a8bf3e/users/9861cfc554b54667854755a19c0be75a/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:53.771560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871e9fe-7d18-4a7a-b86f-698ab633e5e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=453aa10dbf2d4aeb81c81a8169a8bf3e, user_id=9861cfc554b54667854755a19c0be75a, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:53.783947 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 908/1817] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:53 2022] POST /identity/v3/projects => generated 373 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:53.784773 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871e9fe-7d18-4a7a-b86f-698ab633e5e5 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:53.790066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32d3e9-2e41-4cca-891c-a49fd8523678 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:53.829187 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 910/1818] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:53 2022] PUT /identity/v3/projects/453aa10dbf2d4aeb81c81a8169a8bf3e/users/9861cfc554b54667854755a19c0be75a/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:12:53.843510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfb7da-d9e9-4043-9fbf-a49143ff6029 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:53.844705 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfb7da-d9e9-4043-9fbf-a49143ff6029 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:53.844705 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfb7da-d9e9-4043-9fbf-a49143ff6029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:53.944623 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-09cfb7da-d9e9-4043-9fbf-a49143ff6029 None None] MFA Rules not processed for user `9861cfc554b54667854755a19c0be75a`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:53.953182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-09cfb7da-d9e9-4043-9fbf-a49143ff6029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:53.979082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:54.214776 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32d3e9-2e41-4cca-891c-a49fd8523678 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:54.287640 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 911/1819] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:53 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 445 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:54.413326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfb7da-d9e9-4043-9fbf-a49143ff6029 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:54.415340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-09cfb7da-d9e9-4043-9fbf-a49143ff6029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:54.445789 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e302067-c8ff-41ef-93cd-fa0eb010e130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:54.446445 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e302067-c8ff-41ef-93cd-fa0eb010e130 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:54.446730 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e302067-c8ff-41ef-93cd-fa0eb010e130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:54.447005 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e302067-c8ff-41ef-93cd-fa0eb010e130 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:54.452761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e302067-c8ff-41ef-93cd-fa0eb010e130 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:54.455034 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e302067-c8ff-41ef-93cd-fa0eb010e130 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:54.506180 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 909/1820] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:53 2022] POST /identity/v3/users => generated 293 bytes in 717 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:54.508520 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e302067-c8ff-41ef-93cd-fa0eb010e130 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:54.511038 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-3e302067-c8ff-41ef-93cd-fa0eb010e130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:54.625538 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfb7da-d9e9-4043-9fbf-a49143ff6029 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:54.638843 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d4858-f787-4868-9825-d9a933753694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:54.639775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d4858-f787-4868-9825-d9a933753694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:54.640401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d4858-f787-4868-9825-d9a933753694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:54.640796 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d4858-f787-4868-9825-d9a933753694 admin admin] PATH_INFO: `/v3/projects/94e7f861437f4633a71bed1c275d6897` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:54.642468 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d4858-f787-4868-9825-d9a933753694 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94e7f861437f4633a71bed1c275d6897)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:54.645224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d4858-f787-4868-9825-d9a933753694 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:54.718554 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:54.726144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e302067-c8ff-41ef-93cd-fa0eb010e130 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:54.737045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99236e41-f158-446d-9df5-1604b07a4b1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:54.738308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99236e41-f158-446d-9df5-1604b07a4b1d service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:54.738823 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99236e41-f158-446d-9df5-1604b07a4b1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:54.739785 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99236e41-f158-446d-9df5-1604b07a4b1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:54.757691 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99236e41-f158-446d-9df5-1604b07a4b1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:54.759619 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99236e41-f158-446d-9df5-1604b07a4b1d service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:54.773470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 910/1821] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:54 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:54.787597 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99236e41-f158-446d-9df5-1604b07a4b1d service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:54.793761 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1b9d4858-f787-4868-9825-d9a933753694 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94e7f861437f4633a71bed1c275d6897', 'request_id': 'req-1b9d4858-f787-4868-9825-d9a933753694'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:54.814826 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99236e41-f158-446d-9df5-1604b07a4b1d service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:54.817684 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1b9d4858-f787-4868-9825-d9a933753694 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94e7f861437f4633a71bed1c275d6897', 'request_id': 'req-1b9d4858-f787-4868-9825-d9a933753694'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:54.834073 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1b9d4858-f787-4868-9825-d9a933753694 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94e7f861437f4633a71bed1c275d6897', 'request_id': 'req-1b9d4858-f787-4868-9825-d9a933753694'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:54.867700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1b9d4858-f787-4868-9825-d9a933753694 admin admin] The token cache is being invalidate because project 94e7f861437f4633a71bed1c275d6897 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:12:54.870149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1b9d4858-f787-4868-9825-d9a933753694 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:54.875224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1b9d4858-f787-4868-9825-d9a933753694 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:54.879691 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 912/1822] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:12:54 2022] DELETE /identity/v3/projects/94e7f861437f4633a71bed1c275d6897 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:54.886479 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d4858-f787-4868-9825-d9a933753694 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:54.893890 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-1b9d4858-f787-4868-9825-d9a933753694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:54.991970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeef4f1-0a88-43f6-b582-647addda1c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:54.992756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeef4f1-0a88-43f6-b582-647addda1c40 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:54.993059 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeef4f1-0a88-43f6-b582-647addda1c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:54.993325 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeef4f1-0a88-43f6-b582-647addda1c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:54.994209 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeef4f1-0a88-43f6-b582-647addda1c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:54.996281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeef4f1-0a88-43f6-b582-647addda1c40 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:55.011763 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:55.051973 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 911/1823] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:55.053412 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeef4f1-0a88-43f6-b582-647addda1c40 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:55.063144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-dbeef4f1-0a88-43f6-b582-647addda1c40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:55.108128 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeef4f1-0a88-43f6-b582-647addda1c40 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:55.135576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a477388-8168-47f1-ae8a-5a235c13ad71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:55.136758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a477388-8168-47f1-ae8a-5a235c13ad71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:55.138910 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a477388-8168-47f1-ae8a-5a235c13ad71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:55.142539 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a477388-8168-47f1-ae8a-5a235c13ad71 admin admin] PATH_INFO: `/v3/projects/23e5f9aab63542f99fda007487f5b109/users/2a7822281eed4dc4b103f25e30bcc48a/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:55.158390 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d4858-f787-4868-9825-d9a933753694 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:55.183795 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9238717-b1f7-47c6-877e-e3313352dfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:55.186260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9238717-b1f7-47c6-877e-e3313352dfda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:55.187029 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9238717-b1f7-47c6-877e-e3313352dfda admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:55.189836 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9238717-b1f7-47c6-877e-e3313352dfda admin admin] PATH_INFO: `/v3/users/a05f43e064124f518c4b614586ec2c4f` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:55.194132 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9238717-b1f7-47c6-877e-e3313352dfda admin admin] RBAC: Authorizing `identity:delete_user(user_id=a05f43e064124f518c4b614586ec2c4f)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:55.199067 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9238717-b1f7-47c6-877e-e3313352dfda admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:55.202196 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a477388-8168-47f1-ae8a-5a235c13ad71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23e5f9aab63542f99fda007487f5b109, user_id=2a7822281eed4dc4b103f25e30bcc48a, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:55.220669 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a477388-8168-47f1-ae8a-5a235c13ad71 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:55.266525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 912/1824] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:55 2022] PUT /identity/v3/projects/23e5f9aab63542f99fda007487f5b109/users/2a7822281eed4dc4b103f25e30bcc48a/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:55.267646 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a477388-8168-47f1-ae8a-5a235c13ad71 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:55.384507 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-f9238717-b1f7-47c6-877e-e3313352dfda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a05f43e064124f518c4b614586ec2c4f'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:55.427091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-f9238717-b1f7-47c6-877e-e3313352dfda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a05f43e064124f518c4b614586ec2c4f'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:55.446238 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-f9238717-b1f7-47c6-877e-e3313352dfda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a05f43e064124f518c4b614586ec2c4f'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:55.460556 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-f9238717-b1f7-47c6-877e-e3313352dfda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a05f43e064124f518c4b614586ec2c4f'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:55.462161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-f9238717-b1f7-47c6-877e-e3313352dfda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a05f43e064124f518c4b614586ec2c4f'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:55.475754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a477388-8168-47f1-ae8a-5a235c13ad71 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:55.476526 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 913/1825] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:12:54 2022] DELETE /identity/v3/users/a05f43e064124f518c4b614586ec2c4f => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:55.477918 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9238717-b1f7-47c6-877e-e3313352dfda admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:55.479783 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f9238717-b1f7-47c6-877e-e3313352dfda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:55.644163 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9238717-b1f7-47c6-877e-e3313352dfda admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:55.675293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bf7d0-7793-4404-8e5b-65c066298c68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:55.676168 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bf7d0-7793-4404-8e5b-65c066298c68 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:55.676456 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bf7d0-7793-4404-8e5b-65c066298c68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:55.676732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bf7d0-7793-4404-8e5b-65c066298c68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:55.680677 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-549bf7d0-7793-4404-8e5b-65c066298c68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:55.695914 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c913b5-05fd-4d1f-a45a-2c8b183cbf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:55.697125 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c913b5-05fd-4d1f-a45a-2c8b183cbf19 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:55.698198 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c913b5-05fd-4d1f-a45a-2c8b183cbf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:55.698698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c913b5-05fd-4d1f-a45a-2c8b183cbf19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:55.699736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c913b5-05fd-4d1f-a45a-2c8b183cbf19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:55.707595 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c913b5-05fd-4d1f-a45a-2c8b183cbf19 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:55.730925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 913/1826] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 464 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:55.735638 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c913b5-05fd-4d1f-a45a-2c8b183cbf19 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:55.739392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-e8c913b5-05fd-4d1f-a45a-2c8b183cbf19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:55.766026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:55.779398 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c913b5-05fd-4d1f-a45a-2c8b183cbf19 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:55.805240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fcd2a3-7810-4110-9af4-ceeb87f2ebc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:55.815875 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcd2a3-7810-4110-9af4-ceeb87f2ebc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:55.819099 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcd2a3-7810-4110-9af4-ceeb87f2ebc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:55.819401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcd2a3-7810-4110-9af4-ceeb87f2ebc8 admin admin] PATH_INFO: `/v3/projects/23e5f9aab63542f99fda007487f5b109/users/2a7822281eed4dc4b103f25e30bcc48a/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:55.840734 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fcd2a3-7810-4110-9af4-ceeb87f2ebc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23e5f9aab63542f99fda007487f5b109, user_id=2a7822281eed4dc4b103f25e30bcc48a, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:55.859266 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fcd2a3-7810-4110-9af4-ceeb87f2ebc8 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:55.907873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 914/1827] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:55 2022] PUT /identity/v3/projects/23e5f9aab63542f99fda007487f5b109/users/2a7822281eed4dc4b103f25e30bcc48a/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:55.945292 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549bf7d0-7793-4404-8e5b-65c066298c68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:55.955526 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549bf7d0-7793-4404-8e5b-65c066298c68 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:55.981804 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 914/1828] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:55 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:12:56.798569 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676be885-7ede-4161-bc8b-9d1bfdce9ae6 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:56.803434 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676be885-7ede-4161-bc8b-9d1bfdce9ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:56.804816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676be885-7ede-4161-bc8b-9d1bfdce9ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:57.014232 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-676be885-7ede-4161-bc8b-9d1bfdce9ae6 None None] MFA Rules not processed for user `e0c76478b6694782b896cdf9cda32d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:57.019268 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-676be885-7ede-4161-bc8b-9d1bfdce9ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:57.268568 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 915/1829] 10.222.0.73 () {58 vars in 1009 bytes} [Mon Aug 29 20:12:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:57.452935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bf7d0-7793-4404-8e5b-65c066298c68 service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:57.455376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-549bf7d0-7793-4404-8e5b-65c066298c68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:57.464559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676be885-7ede-4161-bc8b-9d1bfdce9ae6 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:57.472244 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-676be885-7ede-4161-bc8b-9d1bfdce9ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:57.484061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bf7d0-7793-4404-8e5b-65c066298c68 service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:57.492372 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:57.660541 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da779847-4966-4c69-99e4-2f66588423aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:57.661489 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da779847-4966-4c69-99e4-2f66588423aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:57.661948 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da779847-4966-4c69-99e4-2f66588423aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:57.662371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da779847-4966-4c69-99e4-2f66588423aa admin admin] PATH_INFO: `/v3/projects/a1a0c81cfa1042d49e2b6b11690e63db` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:57.665189 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da779847-4966-4c69-99e4-2f66588423aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1a0c81cfa1042d49e2b6b11690e63db)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:57.668448 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da779847-4966-4c69-99e4-2f66588423aa admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:57.774968 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676be885-7ede-4161-bc8b-9d1bfdce9ae6 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:57.782507 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2756e-c83e-4c4e-a18c-6a22b3b4f531 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92e30da8f394f3abc19b9e86ef126e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:57.783152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2756e-c83e-4c4e-a18c-6a22b3b4f531 service swift] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:57.783440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2756e-c83e-4c4e-a18c-6a22b3b4f531 service swift] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:57.783826 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2756e-c83e-4c4e-a18c-6a22b3b4f531 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:57.798221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2756e-c83e-4c4e-a18c-6a22b3b4f531 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:57.800078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2756e-c83e-4c4e-a18c-6a22b3b4f531 service swift] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:57.833437 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 916/1830] 10.222.0.73 () {60 vars in 1384 bytes} [Mon Aug 29 20:12:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:12:57.844650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-da779847-4966-4c69-99e4-2f66588423aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1a0c81cfa1042d49e2b6b11690e63db', 'request_id': 'req-da779847-4966-4c69-99e4-2f66588423aa'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:57.860387 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-da779847-4966-4c69-99e4-2f66588423aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1a0c81cfa1042d49e2b6b11690e63db', 'request_id': 'req-da779847-4966-4c69-99e4-2f66588423aa'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:57.879837 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-da779847-4966-4c69-99e4-2f66588423aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1a0c81cfa1042d49e2b6b11690e63db', 'request_id': 'req-da779847-4966-4c69-99e4-2f66588423aa'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:57.914210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-da779847-4966-4c69-99e4-2f66588423aa admin admin] The token cache is being invalidate because project a1a0c81cfa1042d49e2b6b11690e63db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:12:57.914796 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-da779847-4966-4c69-99e4-2f66588423aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:57.916015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-da779847-4966-4c69-99e4-2f66588423aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:57.919320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 915/1831] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:12:57 2022] DELETE /identity/v3/projects/a1a0c81cfa1042d49e2b6b11690e63db => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:12:57.929168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df216cb-1405-47c4-b45b-6bb24754d617 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:57.929451 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df216cb-1405-47c4-b45b-6bb24754d617 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:57.929838 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df216cb-1405-47c4-b45b-6bb24754d617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:58.072876 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-5df216cb-1405-47c4-b45b-6bb24754d617 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:58.080890 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-5df216cb-1405-47c4-b45b-6bb24754d617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:58.200637 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:58.274400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 917/1832] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:58.276111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b6ed8-b20d-41f3-888f-a4d5e2923712 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:58.276401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b6ed8-b20d-41f3-888f-a4d5e2923712 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:58.276681 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b6ed8-b20d-41f3-888f-a4d5e2923712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:58.322936 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-887b6ed8-b20d-41f3-888f-a4d5e2923712 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:58.324207 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-887b6ed8-b20d-41f3-888f-a4d5e2923712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:58.337313 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 916/1833] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:58.344947 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df216cb-1405-47c4-b45b-6bb24754d617 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:58.381396 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df216cb-1405-47c4-b45b-6bb24754d617 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:58.390622 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312db51-00af-40d0-b542-3e548d07c94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:58.391535 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312db51-00af-40d0-b542-3e548d07c94c admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:58.391965 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312db51-00af-40d0-b542-3e548d07c94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:58.392371 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312db51-00af-40d0-b542-3e548d07c94c admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:58.393319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4312db51-00af-40d0-b542-3e548d07c94c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:12:58.393627 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4312db51-00af-40d0-b542-3e548d07c94c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:58.395956 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4312db51-00af-40d0-b542-3e548d07c94c admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:58.441646 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 918/1834] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:12:58.447887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b6ed8-b20d-41f3-888f-a4d5e2923712 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:58.459107 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b6ed8-b20d-41f3-888f-a4d5e2923712 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:58.490004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d3681-c79b-4e26-8f30-8ee1ef353bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:58.493558 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d3681-c79b-4e26-8f30-8ee1ef353bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:58.495902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d3681-c79b-4e26-8f30-8ee1ef353bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:58.498843 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d3681-c79b-4e26-8f30-8ee1ef353bd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:58.504794 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d3681-c79b-4e26-8f30-8ee1ef353bd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:58.520678 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d3681-c79b-4e26-8f30-8ee1ef353bd2 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:58.642360 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 917/1835] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:12:58 2022] POST /identity/v3/projects => generated 387 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:12:58.649978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312db51-00af-40d0-b542-3e548d07c94c admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:58.676682 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312db51-00af-40d0-b542-3e548d07c94c admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:58.677582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d3681-c79b-4e26-8f30-8ee1ef353bd2 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:58.679655 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-776d3681-c79b-4e26-8f30-8ee1ef353bd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:58.972827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c8009-9082-454b-8fc4-6b3bd3488567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:58.973642 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c8009-9082-454b-8fc4-6b3bd3488567 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:58.973954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c8009-9082-454b-8fc4-6b3bd3488567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:58.974228 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c8009-9082-454b-8fc4-6b3bd3488567 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:58.974888 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c8009-9082-454b-8fc4-6b3bd3488567 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:58.980026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c8009-9082-454b-8fc4-6b3bd3488567 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:59.015835 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d3681-c79b-4e26-8f30-8ee1ef353bd2 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:59.037379 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ab7be-163f-4281-91fd-5883baa4ca33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:59.038302 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ab7be-163f-4281-91fd-5883baa4ca33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:59.038746 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ab7be-163f-4281-91fd-5883baa4ca33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:59.039175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ab7be-163f-4281-91fd-5883baa4ca33 admin admin] PATH_INFO: `/v3/users/a5f52c113713454c92757b32f76bb564` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:59.043323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ab7be-163f-4281-91fd-5883baa4ca33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5f52c113713454c92757b32f76bb564)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:59.049649 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ab7be-163f-4281-91fd-5883baa4ca33 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:59.081930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 919/1836] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:12:58 2022] POST /identity/v3/users => generated 294 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:12:59.083201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c8009-9082-454b-8fc4-6b3bd3488567 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:59.116576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c8009-9082-454b-8fc4-6b3bd3488567 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:59.118989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:12:59.136523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a74a96-fbdf-460a-b0fd-f177bc6132bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:59.137536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a74a96-fbdf-460a-b0fd-f177bc6132bc admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:59.138069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a74a96-fbdf-460a-b0fd-f177bc6132bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:59.139764 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a74a96-fbdf-460a-b0fd-f177bc6132bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:59.141510 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a74a96-fbdf-460a-b0fd-f177bc6132bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:59.146109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a74a96-fbdf-460a-b0fd-f177bc6132bc admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:59.175648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 920/1837] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:59.183109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a74a96-fbdf-460a-b0fd-f177bc6132bc admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:59.248254 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a74a96-fbdf-460a-b0fd-f177bc6132bc admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:59.267521 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a31f09-f14c-4b3b-a09a-62f4b333b256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:59.268052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a31f09-f14c-4b3b-a09a-62f4b333b256 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:59.268403 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a31f09-f14c-4b3b-a09a-62f4b333b256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:59.268682 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a31f09-f14c-4b3b-a09a-62f4b333b256 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:59.269352 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a31f09-f14c-4b3b-a09a-62f4b333b256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:59.278007 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a31f09-f14c-4b3b-a09a-62f4b333b256 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:59.278246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1d8ab7be-163f-4281-91fd-5883baa4ca33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5f52c113713454c92757b32f76bb564'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:59.317100 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 921/1838] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:12:59.319634 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a31f09-f14c-4b3b-a09a-62f4b333b256 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:59.348100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1d8ab7be-163f-4281-91fd-5883baa4ca33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5f52c113713454c92757b32f76bb564'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:59.370725 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1d8ab7be-163f-4281-91fd-5883baa4ca33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5f52c113713454c92757b32f76bb564'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:59.383631 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a31f09-f14c-4b3b-a09a-62f4b333b256 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:12:59.402723 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10ef96-30a0-4bc4-9979-a39433ba706b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:12:59.403481 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10ef96-30a0-4bc4-9979-a39433ba706b admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:59.403953 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10ef96-30a0-4bc4-9979-a39433ba706b admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:59.404290 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10ef96-30a0-4bc4-9979-a39433ba706b admin admin] PATH_INFO: `/v3/projects/c62c003ec8c8434189fbad7a9ab51db6/users/79744f5b6c5543878f08381b61e24f63/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:59.411637 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1d8ab7be-163f-4281-91fd-5883baa4ca33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5f52c113713454c92757b32f76bb564'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:59.415710 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1d8ab7be-163f-4281-91fd-5883baa4ca33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5f52c113713454c92757b32f76bb564'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:12:59.442997 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 918/1839] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:12:58 2022] DELETE /identity/v3/users/a5f52c113713454c92757b32f76bb564 => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:59.460192 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d5607-5ecb-4dbe-a7e4-81fed1fe5e4e None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:59.462152 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d5607-5ecb-4dbe-a7e4-81fed1fe5e4e None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:59.462506 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d5607-5ecb-4dbe-a7e4-81fed1fe5e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:59.489419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f10ef96-30a0-4bc4-9979-a39433ba706b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c62c003ec8c8434189fbad7a9ab51db6, user_id=79744f5b6c5543878f08381b61e24f63, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:12:59.494711 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f10ef96-30a0-4bc4-9979-a39433ba706b admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:12:59.518132 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-c52d5607-5ecb-4dbe-a7e4-81fed1fe5e4e None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:59.523678 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c52d5607-5ecb-4dbe-a7e4-81fed1fe5e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:59.549591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 922/1840] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:12:59 2022] PUT /identity/v3/projects/c62c003ec8c8434189fbad7a9ab51db6/users/79744f5b6c5543878f08381b61e24f63/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:12:59.563960 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279eab19-9f7b-4b49-9afe-7739af551207 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:12:59.564369 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279eab19-9f7b-4b49-9afe-7739af551207 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:12:59.564751 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279eab19-9f7b-4b49-9afe-7739af551207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:12:59.686806 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-279eab19-9f7b-4b49-9afe-7739af551207 None None] MFA Rules not processed for user `79744f5b6c5543878f08381b61e24f63`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:12:59.688553 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-279eab19-9f7b-4b49-9afe-7739af551207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:59.702143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 919/1841] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:12:59.721501 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d5607-5ecb-4dbe-a7e4-81fed1fe5e4e None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:12:59.724278 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c52d5607-5ecb-4dbe-a7e4-81fed1fe5e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:12:59.978794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:00.021344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 923/1842] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:12:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:13:00.027239 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279eab19-9f7b-4b49-9afe-7739af551207 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:00.039061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-279eab19-9f7b-4b49-9afe-7739af551207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:00.039265 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d5607-5ecb-4dbe-a7e4-81fed1fe5e4e None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:00.057991 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a82be1-db98-4c14-a54f-5edd283fcbca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:00.059618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a82be1-db98-4c14-a54f-5edd283fcbca service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:00.063052 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a82be1-db98-4c14-a54f-5edd283fcbca service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:00.064567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a82be1-db98-4c14-a54f-5edd283fcbca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:00.100432 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279eab19-9f7b-4b49-9afe-7739af551207 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:00.118498 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62d713-c691-420d-9fd0-b2c1fd4b707a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:00.119435 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62d713-c691-420d-9fd0-b2c1fd4b707a admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:00.120104 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62d713-c691-420d-9fd0-b2c1fd4b707a admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:00.120540 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62d713-c691-420d-9fd0-b2c1fd4b707a admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:00.121366 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf62d713-c691-420d-9fd0-b2c1fd4b707a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:00.131398 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf62d713-c691-420d-9fd0-b2c1fd4b707a admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:00.131648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a82be1-db98-4c14-a54f-5edd283fcbca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:00.137039 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a82be1-db98-4c14-a54f-5edd283fcbca service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:00.198982 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 920/1843] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:12:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:13:00.223636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 924/1844] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:13:00 2022] POST /identity/v3/projects => generated 385 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:00.227867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a82be1-db98-4c14-a54f-5edd283fcbca service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:00.249327 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a82be1-db98-4c14-a54f-5edd283fcbca service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:00.348022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:00.456512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f26f63-6f0b-47e0-b4c6-20e9b57bb970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:00.462656 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f26f63-6f0b-47e0-b4c6-20e9b57bb970 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:00.464214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f26f63-6f0b-47e0-b4c6-20e9b57bb970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:00.464730 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f26f63-6f0b-47e0-b4c6-20e9b57bb970 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:00.467117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f26f63-6f0b-47e0-b4c6-20e9b57bb970 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:00.472618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f26f63-6f0b-47e0-b4c6-20e9b57bb970 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:00.560161 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1fb18-a972-4951-b398-099b56863afb None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:00.560723 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1fb18-a972-4951-b398-099b56863afb None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:00.561185 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1fb18-a972-4951-b398-099b56863afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:00.659751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 921/1845] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:13:00 2022] POST /identity/v3/users => generated 299 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:00.669090 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f26f63-6f0b-47e0-b4c6-20e9b57bb970 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:00.716066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-a5b1fb18-a972-4951-b398-099b56863afb None None] MFA Rules not processed for user `64ba58d774674ed7b233ad6416db068a`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:00.718737 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-a5b1fb18-a972-4951-b398-099b56863afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:00.741727 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f26f63-6f0b-47e0-b4c6-20e9b57bb970 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:00.789868 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f36a7-dcab-48ad-82bd-8f976b85c954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:00.794272 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f36a7-dcab-48ad-82bd-8f976b85c954 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:00.795769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f36a7-dcab-48ad-82bd-8f976b85c954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:00.796259 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f36a7-dcab-48ad-82bd-8f976b85c954 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:00.797574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f36a7-dcab-48ad-82bd-8f976b85c954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:00.813619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f36a7-dcab-48ad-82bd-8f976b85c954 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:00.887263 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 922/1846] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:00.893686 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f36a7-dcab-48ad-82bd-8f976b85c954 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:00.976408 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f36a7-dcab-48ad-82bd-8f976b85c954 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:01.007332 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdec9a-51e4-441b-8a6e-039145e08f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:01.008457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdec9a-51e4-441b-8a6e-039145e08f3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:01.008924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdec9a-51e4-441b-8a6e-039145e08f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:01.009493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdec9a-51e4-441b-8a6e-039145e08f3b admin admin] PATH_INFO: `/v3/projects/19a8cc95f77c4e188bb4cd8f8e3b31c3/users/4bcf4eb414bf411c8fe698fafd3a9bca/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:01.064331 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 925/1847] 10.222.0.77 () {58 vars in 1015 bytes} [Mon Aug 29 20:13:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 507 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:01.077686 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bdec9a-51e4-441b-8a6e-039145e08f3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a8cc95f77c4e188bb4cd8f8e3b31c3, user_id=4bcf4eb414bf411c8fe698fafd3a9bca, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:01.083446 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bdec9a-51e4-441b-8a6e-039145e08f3b admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:01.151514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1fb18-a972-4951-b398-099b56863afb None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:01.157228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 923/1848] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:00 2022] PUT /identity/v3/projects/19a8cc95f77c4e188bb4cd8f8e3b31c3/users/4bcf4eb414bf411c8fe698fafd3a9bca/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:01.167716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdec9a-51e4-441b-8a6e-039145e08f3b admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:01.198299 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1fb18-a972-4951-b398-099b56863afb None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:01.431722 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca898f7e-9e76-4cea-a82b-8030bf3eb208 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:01.433558 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca898f7e-9e76-4cea-a82b-8030bf3eb208 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:01.434357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca898f7e-9e76-4cea-a82b-8030bf3eb208 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:01.435113 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca898f7e-9e76-4cea-a82b-8030bf3eb208 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:01.459309 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca898f7e-9e76-4cea-a82b-8030bf3eb208 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:01.461163 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca898f7e-9e76-4cea-a82b-8030bf3eb208 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:01.482236 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 926/1849] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:13:01.487138 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca898f7e-9e76-4cea-a82b-8030bf3eb208 service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:01.492104 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-ca898f7e-9e76-4cea-a82b-8030bf3eb208 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:01.560920 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdec9a-51e4-441b-8a6e-039145e08f3b admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:01.584350 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db44929-c75e-4dc3-a89a-88eca5a2debf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:01.595224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db44929-c75e-4dc3-a89a-88eca5a2debf admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:01.595542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db44929-c75e-4dc3-a89a-88eca5a2debf admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:01.595817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db44929-c75e-4dc3-a89a-88eca5a2debf admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:01.599222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db44929-c75e-4dc3-a89a-88eca5a2debf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:01.604873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db44929-c75e-4dc3-a89a-88eca5a2debf admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:01.654438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 924/1850] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 493 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:01.666121 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db44929-c75e-4dc3-a89a-88eca5a2debf admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:01.673903 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca898f7e-9e76-4cea-a82b-8030bf3eb208 service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:01.690651 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e849bfa-3e7d-4a90-afc4-dccbb524c576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:01.692199 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e849bfa-3e7d-4a90-afc4-dccbb524c576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:01.692860 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e849bfa-3e7d-4a90-afc4-dccbb524c576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:01.694324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e849bfa-3e7d-4a90-afc4-dccbb524c576 admin admin] PATH_INFO: `/v3/projects/a167c5de72034f2e8a657ef35e82da08` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:01.695363 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db44929-c75e-4dc3-a89a-88eca5a2debf admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:01.703255 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e849bfa-3e7d-4a90-afc4-dccbb524c576 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a167c5de72034f2e8a657ef35e82da08)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:01.707633 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e849bfa-3e7d-4a90-afc4-dccbb524c576 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:01.722251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48781544-0f0f-4977-9817-014564b9fc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:01.726251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48781544-0f0f-4977-9817-014564b9fc39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:01.728182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48781544-0f0f-4977-9817-014564b9fc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:01.729267 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48781544-0f0f-4977-9817-014564b9fc39 admin admin] PATH_INFO: `/v3/projects/19a8cc95f77c4e188bb4cd8f8e3b31c3/users/4bcf4eb414bf411c8fe698fafd3a9bca/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:01.740263 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48781544-0f0f-4977-9817-014564b9fc39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a8cc95f77c4e188bb4cd8f8e3b31c3, user_id=4bcf4eb414bf411c8fe698fafd3a9bca, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:01.749180 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48781544-0f0f-4977-9817-014564b9fc39 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:01.793746 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 925/1851] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:01 2022] PUT /identity/v3/projects/19a8cc95f77c4e188bb4cd8f8e3b31c3/users/4bcf4eb414bf411c8fe698fafd3a9bca/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:13:01.794047 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:01.808130 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b57e4-1cde-4839-8431-e481c519c1aa None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:01.808449 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b57e4-1cde-4839-8431-e481c519c1aa None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:01.808732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b57e4-1cde-4839-8431-e481c519c1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:01.820539 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8e849bfa-3e7d-4a90-afc4-dccbb524c576 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a167c5de72034f2e8a657ef35e82da08', 'request_id': 'req-8e849bfa-3e7d-4a90-afc4-dccbb524c576'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:01.851645 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8e849bfa-3e7d-4a90-afc4-dccbb524c576 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a167c5de72034f2e8a657ef35e82da08', 'request_id': 'req-8e849bfa-3e7d-4a90-afc4-dccbb524c576'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:01.871636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8e849bfa-3e7d-4a90-afc4-dccbb524c576 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a167c5de72034f2e8a657ef35e82da08', 'request_id': 'req-8e849bfa-3e7d-4a90-afc4-dccbb524c576'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:01.894718 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8e849bfa-3e7d-4a90-afc4-dccbb524c576 admin admin] The token cache is being invalidate because project a167c5de72034f2e8a657ef35e82da08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:13:01.895469 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8e849bfa-3e7d-4a90-afc4-dccbb524c576 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:01.896333 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8e849bfa-3e7d-4a90-afc4-dccbb524c576 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:01.901166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-fc6b57e4-1cde-4839-8431-e481c519c1aa None None] MFA Rules not processed for user `4bcf4eb414bf411c8fe698fafd3a9bca`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:01.901806 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 927/1852] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:13:01 2022] DELETE /identity/v3/projects/a167c5de72034f2e8a657ef35e82da08 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:01.902936 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e849bfa-3e7d-4a90-afc4-dccbb524c576 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:01.903221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-fc6b57e4-1cde-4839-8431-e481c519c1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:01.904941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-8e849bfa-3e7d-4a90-afc4-dccbb524c576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:02.076071 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:02.079459 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e849bfa-3e7d-4a90-afc4-dccbb524c576 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:02.105480 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 926/1853] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:01 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:02.105698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a50f0-e1b8-46e2-8d0c-1085e385cf84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:02.105698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a50f0-e1b8-46e2-8d0c-1085e385cf84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:02.105698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a50f0-e1b8-46e2-8d0c-1085e385cf84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:02.105698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a50f0-e1b8-46e2-8d0c-1085e385cf84 admin admin] PATH_INFO: `/v3/users/df4115d2479046e1b3f2a328ee7828cf` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:02.106246 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a50f0-e1b8-46e2-8d0c-1085e385cf84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df4115d2479046e1b3f2a328ee7828cf)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:02.111385 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a50f0-e1b8-46e2-8d0c-1085e385cf84 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:02.115250 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b57e4-1cde-4839-8431-e481c519c1aa None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:02.116828 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-fc6b57e4-1cde-4839-8431-e481c519c1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:02.269412 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-0f0a50f0-e1b8-46e2-8d0c-1085e385cf84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df4115d2479046e1b3f2a328ee7828cf'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:02.284935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b57e4-1cde-4839-8431-e481c519c1aa None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:02.289388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545bb25-2ebd-4002-b918-955427cf0c5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:02.290451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545bb25-2ebd-4002-b918-955427cf0c5c service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:02.290679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-0f0a50f0-e1b8-46e2-8d0c-1085e385cf84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df4115d2479046e1b3f2a328ee7828cf'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:02.291183 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545bb25-2ebd-4002-b918-955427cf0c5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:02.291613 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545bb25-2ebd-4002-b918-955427cf0c5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:02.323201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6545bb25-2ebd-4002-b918-955427cf0c5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:02.325070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6545bb25-2ebd-4002-b918-955427cf0c5c service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:02.334700 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-0f0a50f0-e1b8-46e2-8d0c-1085e385cf84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df4115d2479046e1b3f2a328ee7828cf'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:02.372351 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 927/1854] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:02 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:13:02.383525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-0f0a50f0-e1b8-46e2-8d0c-1085e385cf84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df4115d2479046e1b3f2a328ee7828cf'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:02.384756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-0f0a50f0-e1b8-46e2-8d0c-1085e385cf84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df4115d2479046e1b3f2a328ee7828cf'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:02.406053 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 928/1855] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:13:01 2022] DELETE /identity/v3/users/df4115d2479046e1b3f2a328ee7828cf => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:02.423969 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545bb25-2ebd-4002-b918-955427cf0c5c service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:02.428812 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-6545bb25-2ebd-4002-b918-955427cf0c5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:02.488164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545bb25-2ebd-4002-b918-955427cf0c5c service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:02.790109 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e189c2-8fd2-43cb-a6ad-10f596e5594d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:02.790109 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e189c2-8fd2-43cb-a6ad-10f596e5594d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:02.790109 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e189c2-8fd2-43cb-a6ad-10f596e5594d admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:02.790109 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e189c2-8fd2-43cb-a6ad-10f596e5594d admin admin] PATH_INFO: `/v3/users/79744f5b6c5543878f08381b61e24f63` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:02.790897 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e189c2-8fd2-43cb-a6ad-10f596e5594d admin admin] RBAC: Authorizing `identity:delete_user(user_id=79744f5b6c5543878f08381b61e24f63)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:02.797193 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e189c2-8fd2-43cb-a6ad-10f596e5594d admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:02.959316 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-93e189c2-8fd2-43cb-a6ad-10f596e5594d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79744f5b6c5543878f08381b61e24f63'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:02.991266 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-93e189c2-8fd2-43cb-a6ad-10f596e5594d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79744f5b6c5543878f08381b61e24f63'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:03.006761 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-93e189c2-8fd2-43cb-a6ad-10f596e5594d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79744f5b6c5543878f08381b61e24f63'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:03.015198 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a50f0-e1b8-46e2-8d0c-1085e385cf84 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:03.016671 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-0f0a50f0-e1b8-46e2-8d0c-1085e385cf84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:03.046079 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-93e189c2-8fd2-43cb-a6ad-10f596e5594d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79744f5b6c5543878f08381b61e24f63'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:03.049543 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-93e189c2-8fd2-43cb-a6ad-10f596e5594d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79744f5b6c5543878f08381b61e24f63'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:03.079413 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a50f0-e1b8-46e2-8d0c-1085e385cf84 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:03.110695 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 928/1856] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:13:02 2022] DELETE /identity/v3/users/79744f5b6c5543878f08381b61e24f63 => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:03.120932 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a076c-851e-41a6-b50d-2376be433816 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:03.123846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06612a3-1227-4d1d-ba1a-ad12ebfa7989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:03.125322 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a076c-851e-41a6-b50d-2376be433816 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:03.126900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a076c-851e-41a6-b50d-2376be433816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:03.128703 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06612a3-1227-4d1d-ba1a-ad12ebfa7989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:03.131380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06612a3-1227-4d1d-ba1a-ad12ebfa7989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:03.131868 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06612a3-1227-4d1d-ba1a-ad12ebfa7989 admin admin] PATH_INFO: `/v3/users/05e24d04aa604053b8dda62e67ec746b` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:03.138486 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06612a3-1227-4d1d-ba1a-ad12ebfa7989 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05e24d04aa604053b8dda62e67ec746b)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:03.151930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06612a3-1227-4d1d-ba1a-ad12ebfa7989 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:03.189344 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-ed6a076c-851e-41a6-b50d-2376be433816 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:03.190788 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ed6a076c-851e-41a6-b50d-2376be433816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:03.203061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:03.233890 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:DELETE=21,keystone:SELECT=77,keystone:INSERT=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:03.274691 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a06612a3-1227-4d1d-ba1a-ad12ebfa7989 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05e24d04aa604053b8dda62e67ec746b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:03.298205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a06612a3-1227-4d1d-ba1a-ad12ebfa7989 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05e24d04aa604053b8dda62e67ec746b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:03.343814 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a06612a3-1227-4d1d-ba1a-ad12ebfa7989 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05e24d04aa604053b8dda62e67ec746b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:03.348541 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 929/1857] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:03.363352 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a06612a3-1227-4d1d-ba1a-ad12ebfa7989 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05e24d04aa604053b8dda62e67ec746b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:03.364293 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a06612a3-1227-4d1d-ba1a-ad12ebfa7989 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05e24d04aa604053b8dda62e67ec746b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:03.376759 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 929/1858] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:13:03 2022] DELETE /identity/v3/users/05e24d04aa604053b8dda62e67ec746b => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:13:03.379823 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaee570-e00b-4224-ad31-2cd17ee33596 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:03.380171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaee570-e00b-4224-ad31-2cd17ee33596 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:03.380455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaee570-e00b-4224-ad31-2cd17ee33596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:03.397784 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06612a3-1227-4d1d-ba1a-ad12ebfa7989 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:03.400559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-a06612a3-1227-4d1d-ba1a-ad12ebfa7989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:03.429351 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-cbaee570-e00b-4224-ad31-2cd17ee33596 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:03.435589 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-cbaee570-e00b-4224-ad31-2cd17ee33596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:03.627996 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06612a3-1227-4d1d-ba1a-ad12ebfa7989 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:03.638080 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d2f13-b0e4-482d-ae7b-0ec523582aea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:03.639257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d2f13-b0e4-482d-ae7b-0ec523582aea service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:03.639692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d2f13-b0e4-482d-ae7b-0ec523582aea service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:03.640158 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d2f13-b0e4-482d-ae7b-0ec523582aea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:03.642403 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-b14d2f13-b0e4-482d-ae7b-0ec523582aea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:03.677759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 930/1859] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:03.761257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d2f13-b0e4-482d-ae7b-0ec523582aea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:03.763639 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d2f13-b0e4-482d-ae7b-0ec523582aea service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:03.777548 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 930/1860] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:13:03.782989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaee570-e00b-4224-ad31-2cd17ee33596 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:03.802934 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaee570-e00b-4224-ad31-2cd17ee33596 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:03.812890 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8f7e2-d910-49ea-a1ea-607dfabfac98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:03.813530 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8f7e2-d910-49ea-a1ea-607dfabfac98 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:03.813923 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8f7e2-d910-49ea-a1ea-607dfabfac98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:03.814174 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8f7e2-d910-49ea-a1ea-607dfabfac98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:03.835697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8f7e2-d910-49ea-a1ea-607dfabfac98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:03.837173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8f7e2-d910-49ea-a1ea-607dfabfac98 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:03.862901 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 931/1861] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:13:04.364225 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fd256-b542-47ee-97e7-7c52128ea433 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:04.364841 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fd256-b542-47ee-97e7-7c52128ea433 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:04.365116 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fd256-b542-47ee-97e7-7c52128ea433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:04.517343 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-0a8fd256-b542-47ee-97e7-7c52128ea433 None None] MFA Rules not processed for user `2a7822281eed4dc4b103f25e30bcc48a`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:04.520006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-0a8fd256-b542-47ee-97e7-7c52128ea433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:04.698487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:04.769987 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 931/1862] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:04.786171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8f7e2-d910-49ea-a1ea-607dfabfac98 service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:04.791801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c7c8f7e2-d910-49ea-a1ea-607dfabfac98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:04.897234 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fd256-b542-47ee-97e7-7c52128ea433 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:04.898776 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-0a8fd256-b542-47ee-97e7-7c52128ea433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:04.948775 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fd256-b542-47ee-97e7-7c52128ea433 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:04.993217 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b68c9a-2940-4edc-b600-251c66c152eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:05.000464 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b68c9a-2940-4edc-b600-251c66c152eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:05.002009 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b68c9a-2940-4edc-b600-251c66c152eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:05.004782 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b68c9a-2940-4edc-b600-251c66c152eb admin admin] PATH_INFO: `/v3/projects/bf833eda0ba94fbeaf7f2c60ae84fb0d` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:05.011286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b68c9a-2940-4edc-b600-251c66c152eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf833eda0ba94fbeaf7f2c60ae84fb0d)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:05.018178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b68c9a-2940-4edc-b600-251c66c152eb admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:05.069270 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8f7e2-d910-49ea-a1ea-607dfabfac98 service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:05.084022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f3b0a-7d45-4e51-9d99-8dd9dcc7d30a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:05.084830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f3b0a-7d45-4e51-9d99-8dd9dcc7d30a service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:05.085307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f3b0a-7d45-4e51-9d99-8dd9dcc7d30a service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:05.085546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f3b0a-7d45-4e51-9d99-8dd9dcc7d30a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:05.098642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-623f3b0a-7d45-4e51-9d99-8dd9dcc7d30a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:05.152177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-19b68c9a-2940-4edc-b600-251c66c152eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf833eda0ba94fbeaf7f2c60ae84fb0d', 'request_id': 'req-19b68c9a-2940-4edc-b600-251c66c152eb'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:05.168281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-19b68c9a-2940-4edc-b600-251c66c152eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf833eda0ba94fbeaf7f2c60ae84fb0d', 'request_id': 'req-19b68c9a-2940-4edc-b600-251c66c152eb'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:05.199935 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-19b68c9a-2940-4edc-b600-251c66c152eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf833eda0ba94fbeaf7f2c60ae84fb0d', 'request_id': 'req-19b68c9a-2940-4edc-b600-251c66c152eb'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:05.229744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-19b68c9a-2940-4edc-b600-251c66c152eb admin admin] The token cache is being invalidate because project bf833eda0ba94fbeaf7f2c60ae84fb0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:13:05.230090 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-19b68c9a-2940-4edc-b600-251c66c152eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:05.230816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-19b68c9a-2940-4edc-b600-251c66c152eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:05.247289 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 932/1863] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:13:04 2022] DELETE /identity/v3/projects/bf833eda0ba94fbeaf7f2c60ae84fb0d => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:13:05.256570 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16523dc-ed1a-413e-862b-7482fdad50ae None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:05.257221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16523dc-ed1a-413e-862b-7482fdad50ae None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:05.257934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16523dc-ed1a-413e-862b-7482fdad50ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:05.364607 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-d16523dc-ed1a-413e-862b-7482fdad50ae None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:05.371612 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d16523dc-ed1a-413e-862b-7482fdad50ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:05.414526 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f3b0a-7d45-4e51-9d99-8dd9dcc7d30a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:05.419972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f3b0a-7d45-4e51-9d99-8dd9dcc7d30a service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:05.433306 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:05.467539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 932/1864] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:13:05.550762 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dffa53-9415-41cc-aa0d-a6f10df80403 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:05.551044 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 933/1865] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:05.552441 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dffa53-9415-41cc-aa0d-a6f10df80403 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:05.553141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dffa53-9415-41cc-aa0d-a6f10df80403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:05.665345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-d2dffa53-9415-41cc-aa0d-a6f10df80403 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:05.668148 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-d2dffa53-9415-41cc-aa0d-a6f10df80403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:05.694750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16523dc-ed1a-413e-862b-7482fdad50ae None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:05.696962 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d16523dc-ed1a-413e-862b-7482fdad50ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:05.704094 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 933/1866] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:13:05.716010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d7710-280c-4045-aa98-2e431bcb6f66 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:05.718479 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d7710-280c-4045-aa98-2e431bcb6f66 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:05.721041 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d7710-280c-4045-aa98-2e431bcb6f66 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:05.722571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16523dc-ed1a-413e-862b-7482fdad50ae None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:05.727722 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 934/1867] 10.222.0.73 () {54 vars in 921 bytes} [Mon Aug 29 20:13:05 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:13:05.731878 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d7710-280c-4045-aa98-2e431bcb6f66 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:05.738884 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcacddb-e216-4773-af15-3672ccf92b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:05.739668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcacddb-e216-4773-af15-3672ccf92b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:05.740034 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcacddb-e216-4773-af15-3672ccf92b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:05.740390 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcacddb-e216-4773-af15-3672ccf92b20 admin admin] PATH_INFO: `/v3/projects/c62c003ec8c8434189fbad7a9ab51db6` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:05.742338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcacddb-e216-4773-af15-3672ccf92b20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c62c003ec8c8434189fbad7a9ab51db6)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:05.745410 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcacddb-e216-4773-af15-3672ccf92b20 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:05.825302 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d7710-280c-4045-aa98-2e431bcb6f66 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:05.843973 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb5ad7-7e6f-40c7-b509-30629af0cd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:05.844868 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb5ad7-7e6f-40c7-b509-30629af0cd94 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:05.848134 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb5ad7-7e6f-40c7-b509-30629af0cd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:05.849189 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb5ad7-7e6f-40c7-b509-30629af0cd94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:05.860379 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb5ad7-7e6f-40c7-b509-30629af0cd94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:13:05.861056 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb5ad7-7e6f-40c7-b509-30629af0cd94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:05.866631 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb5ad7-7e6f-40c7-b509-30629af0cd94 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:05.882551 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-bfcacddb-e216-4773-af15-3672ccf92b20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c62c003ec8c8434189fbad7a9ab51db6', 'request_id': 'req-bfcacddb-e216-4773-af15-3672ccf92b20'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:05.901096 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-bfcacddb-e216-4773-af15-3672ccf92b20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c62c003ec8c8434189fbad7a9ab51db6', 'request_id': 'req-bfcacddb-e216-4773-af15-3672ccf92b20'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:05.915018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:05.923934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-bfcacddb-e216-4773-af15-3672ccf92b20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c62c003ec8c8434189fbad7a9ab51db6', 'request_id': 'req-bfcacddb-e216-4773-af15-3672ccf92b20'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:05.929626 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 935/1868] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:13:05.931934 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922f995-ef97-481b-a448-cd2ee56320ab None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:05.932979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922f995-ef97-481b-a448-cd2ee56320ab None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:05.934173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922f995-ef97-481b-a448-cd2ee56320ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:05.955575 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-bfcacddb-e216-4773-af15-3672ccf92b20 admin admin] The token cache is being invalidate because project c62c003ec8c8434189fbad7a9ab51db6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:13:05.955887 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-bfcacddb-e216-4773-af15-3672ccf92b20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:05.957640 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-bfcacddb-e216-4773-af15-3672ccf92b20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:05.964110 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 934/1869] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:13:05 2022] DELETE /identity/v3/projects/c62c003ec8c8434189fbad7a9ab51db6 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:05.964867 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcacddb-e216-4773-af15-3672ccf92b20 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:05.969101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-bfcacddb-e216-4773-af15-3672ccf92b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:06.006242 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-5922f995-ef97-481b-a448-cd2ee56320ab None None] MFA Rules not processed for user `95cf77a92a174ff8b0503baf5c7e63be`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:06.009847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-5922f995-ef97-481b-a448-cd2ee56320ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:06.165975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcacddb-e216-4773-af15-3672ccf92b20 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:06.172957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c625d-0542-4504-8dcb-8b90516bae63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:06.173696 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c625d-0542-4504-8dcb-8b90516bae63 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:06.174111 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c625d-0542-4504-8dcb-8b90516bae63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:06.174394 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c625d-0542-4504-8dcb-8b90516bae63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:06.175037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c625d-0542-4504-8dcb-8b90516bae63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:06.177915 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c625d-0542-4504-8dcb-8b90516bae63 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:06.266268 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 935/1870] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:13:05 2022] POST /identity/v3/projects => generated 375 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:06.268525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c625d-0542-4504-8dcb-8b90516bae63 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:06.275304 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-f88c625d-0542-4504-8dcb-8b90516bae63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:06.330977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 936/1871] 10.222.0.73 () {58 vars in 1018 bytes} [Mon Aug 29 20:13:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:06.338648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922f995-ef97-481b-a448-cd2ee56320ab None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:06.381297 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922f995-ef97-481b-a448-cd2ee56320ab None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:06.492723 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c625d-0542-4504-8dcb-8b90516bae63 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:06.504903 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e45b87-66ca-41a4-90d9-742aeaae3da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:06.505980 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e45b87-66ca-41a4-90d9-742aeaae3da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:06.506412 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e45b87-66ca-41a4-90d9-742aeaae3da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:06.506848 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e45b87-66ca-41a4-90d9-742aeaae3da8 admin admin] PATH_INFO: `/v3/users/4bcf4eb414bf411c8fe698fafd3a9bca` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:06.510731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e45b87-66ca-41a4-90d9-742aeaae3da8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bcf4eb414bf411c8fe698fafd3a9bca)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:06.524933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e45b87-66ca-41a4-90d9-742aeaae3da8 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:06.547868 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef775f63-d799-433c-9692-affecafa2560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:06.550557 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef775f63-d799-433c-9692-affecafa2560 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:06.550892 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef775f63-d799-433c-9692-affecafa2560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:06.551454 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef775f63-d799-433c-9692-affecafa2560 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:06.552283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef775f63-d799-433c-9692-affecafa2560 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:06.554995 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef775f63-d799-433c-9692-affecafa2560 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:06.627773 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 937/1872] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:13:06 2022] POST /identity/v3/users => generated 288 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:06.629268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef775f63-d799-433c-9692-affecafa2560 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:06.631959 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ef775f63-d799-433c-9692-affecafa2560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:06.695240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d1e45b87-66ca-41a4-90d9-742aeaae3da8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bcf4eb414bf411c8fe698fafd3a9bca'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:06.719218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d1e45b87-66ca-41a4-90d9-742aeaae3da8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bcf4eb414bf411c8fe698fafd3a9bca'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:06.763546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef775f63-d799-433c-9692-affecafa2560 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:06.778007 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d1e45b87-66ca-41a4-90d9-742aeaae3da8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bcf4eb414bf411c8fe698fafd3a9bca'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:06.787060 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:06.808087 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad1834-a4a9-43d8-a2a7-3ab0677d3e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:06.811007 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d1e45b87-66ca-41a4-90d9-742aeaae3da8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bcf4eb414bf411c8fe698fafd3a9bca'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:06.811880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d1e45b87-66ca-41a4-90d9-742aeaae3da8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bcf4eb414bf411c8fe698fafd3a9bca'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:06.816997 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad1834-a4a9-43d8-a2a7-3ab0677d3e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:06.817410 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad1834-a4a9-43d8-a2a7-3ab0677d3e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:06.818964 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad1834-a4a9-43d8-a2a7-3ab0677d3e3a admin admin] PATH_INFO: `/v3/projects/f4d9d8ab0f934a3d836e70edb7dbddf5` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:06.826174 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ad1834-a4a9-43d8-a2a7-3ab0677d3e3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4d9d8ab0f934a3d836e70edb7dbddf5)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:06.829737 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 936/1873] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:13:06 2022] DELETE /identity/v3/users/4bcf4eb414bf411c8fe698fafd3a9bca => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:06.834245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e45b87-66ca-41a4-90d9-742aeaae3da8 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:06.835511 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ad1834-a4a9-43d8-a2a7-3ab0677d3e3a admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:06.835718 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d1e45b87-66ca-41a4-90d9-742aeaae3da8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:07.005101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a5ad1834-a4a9-43d8-a2a7-3ab0677d3e3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4d9d8ab0f934a3d836e70edb7dbddf5', 'request_id': 'req-a5ad1834-a4a9-43d8-a2a7-3ab0677d3e3a'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:07.017080 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e45b87-66ca-41a4-90d9-742aeaae3da8 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:07.023364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:07.031093 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a5ad1834-a4a9-43d8-a2a7-3ab0677d3e3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4d9d8ab0f934a3d836e70edb7dbddf5', 'request_id': 'req-a5ad1834-a4a9-43d8-a2a7-3ab0677d3e3a'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:07.067895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a5ad1834-a4a9-43d8-a2a7-3ab0677d3e3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4d9d8ab0f934a3d836e70edb7dbddf5', 'request_id': 'req-a5ad1834-a4a9-43d8-a2a7-3ab0677d3e3a'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:07.111803 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a5ad1834-a4a9-43d8-a2a7-3ab0677d3e3a admin admin] The token cache is being invalidate because project f4d9d8ab0f934a3d836e70edb7dbddf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:13:07.113537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a5ad1834-a4a9-43d8-a2a7-3ab0677d3e3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:07.114693 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a5ad1834-a4a9-43d8-a2a7-3ab0677d3e3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:07.118066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 938/1874] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:13:06 2022] DELETE /identity/v3/projects/f4d9d8ab0f934a3d836e70edb7dbddf5 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:07.122056 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad1834-a4a9-43d8-a2a7-3ab0677d3e3a admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:07.124045 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a5ad1834-a4a9-43d8-a2a7-3ab0677d3e3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:07.381012 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad1834-a4a9-43d8-a2a7-3ab0677d3e3a admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:07.389031 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78291dd-d383-4730-9669-ff725b753e9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:07.390718 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78291dd-d383-4730-9669-ff725b753e9c service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:07.391227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78291dd-d383-4730-9669-ff725b753e9c service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:07.391732 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78291dd-d383-4730-9669-ff725b753e9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:07.398253 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9346bb0-33e2-4f8b-a03a-812111849507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:07.398889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9346bb0-33e2-4f8b-a03a-812111849507 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:07.399172 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9346bb0-33e2-4f8b-a03a-812111849507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:07.399445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9346bb0-33e2-4f8b-a03a-812111849507 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:07.400208 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9346bb0-33e2-4f8b-a03a-812111849507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:07.400987 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c78291dd-d383-4730-9669-ff725b753e9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:07.401943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9346bb0-33e2-4f8b-a03a-812111849507 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:07.433318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78291dd-d383-4730-9669-ff725b753e9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:07.435760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78291dd-d383-4730-9669-ff725b753e9c service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:07.474012 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 939/1875] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:07.474012 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9346bb0-33e2-4f8b-a03a-812111849507 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:07.474890 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 937/1876] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:13:07.474890 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78291dd-d383-4730-9669-ff725b753e9c service nova] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:07.475420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c9346bb0-33e2-4f8b-a03a-812111849507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:07.475528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c78291dd-d383-4730-9669-ff725b753e9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:07.510317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9346bb0-33e2-4f8b-a03a-812111849507 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:07.539067 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a03f57-a5c4-48f2-b452-115fa185f7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:07.539734 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a03f57-a5c4-48f2-b452-115fa185f7d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:07.540014 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a03f57-a5c4-48f2-b452-115fa185f7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:07.549885 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a03f57-a5c4-48f2-b452-115fa185f7d1 admin admin] PATH_INFO: `/v3/users/9861cfc554b54667854755a19c0be75a` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:07.551597 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a03f57-a5c4-48f2-b452-115fa185f7d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9861cfc554b54667854755a19c0be75a)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:07.553526 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a03f57-a5c4-48f2-b452-115fa185f7d1 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:07.640111 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78291dd-d383-4730-9669-ff725b753e9c service nova] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:07.655617 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846530c-84fe-436d-b989-29122218eed8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:07.658880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846530c-84fe-436d-b989-29122218eed8 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:07.660218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846530c-84fe-436d-b989-29122218eed8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:07.660866 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846530c-84fe-436d-b989-29122218eed8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:07.666923 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-0846530c-84fe-436d-b989-29122218eed8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:07.707189 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-03a03f57-a5c4-48f2-b452-115fa185f7d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9861cfc554b54667854755a19c0be75a'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:07.747175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-03a03f57-a5c4-48f2-b452-115fa185f7d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9861cfc554b54667854755a19c0be75a'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:07.764742 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-03a03f57-a5c4-48f2-b452-115fa185f7d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9861cfc554b54667854755a19c0be75a'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:07.765675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:07.780597 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-03a03f57-a5c4-48f2-b452-115fa185f7d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9861cfc554b54667854755a19c0be75a'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:07.782222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-03a03f57-a5c4-48f2-b452-115fa185f7d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9861cfc554b54667854755a19c0be75a'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:07.805378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 940/1877] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:13:07 2022] DELETE /identity/v3/users/9861cfc554b54667854755a19c0be75a => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:07.805378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a03f57-a5c4-48f2-b452-115fa185f7d1 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:07.805896 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-03a03f57-a5c4-48f2-b452-115fa185f7d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:07.866145 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846530c-84fe-436d-b989-29122218eed8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:07.868520 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846530c-84fe-436d-b989-29122218eed8 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:07.883711 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 938/1878] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:13:07.931499 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:07.975909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a03f57-a5c4-48f2-b452-115fa185f7d1 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:07.983496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4215c5-7f19-46cd-b1d1-a650a2afa1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:07.984279 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4215c5-7f19-46cd-b1d1-a650a2afa1cf admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:07.984954 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4215c5-7f19-46cd-b1d1-a650a2afa1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:07.985537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4215c5-7f19-46cd-b1d1-a650a2afa1cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:07.986589 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4215c5-7f19-46cd-b1d1-a650a2afa1cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:07.988580 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4215c5-7f19-46cd-b1d1-a650a2afa1cf admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:08.016522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 941/1879] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:08.016840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846530c-84fe-436d-b989-29122218eed8 service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:08.040815 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846530c-84fe-436d-b989-29122218eed8 service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:08.052426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb6f42-a879-4fd8-88cd-4ae034acf793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:08.053931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb6f42-a879-4fd8-88cd-4ae034acf793 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:08.055054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb6f42-a879-4fd8-88cd-4ae034acf793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:08.055298 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb6f42-a879-4fd8-88cd-4ae034acf793 admin admin] PATH_INFO: `/v3/projects/67fb63da3ed9417b9fc7476a4b822e6a/users/df57699e3cd1494d981ddb451a56c899/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:08.110788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb6f42-a879-4fd8-88cd-4ae034acf793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67fb63da3ed9417b9fc7476a4b822e6a, user_id=df57699e3cd1494d981ddb451a56c899, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:08.127866 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb6f42-a879-4fd8-88cd-4ae034acf793 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:08.170877 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3abb2-8f81-463e-9d0a-eba3ca0dd3e7 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:08.171966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3abb2-8f81-463e-9d0a-eba3ca0dd3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:08.171966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3abb2-8f81-463e-9d0a-eba3ca0dd3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:08.175266 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 939/1880] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:08 2022] PUT /identity/v3/projects/67fb63da3ed9417b9fc7476a4b822e6a/users/df57699e3cd1494d981ddb451a56c899/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:08.230586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-8af3abb2-8f81-463e-9d0a-eba3ca0dd3e7 None None] MFA Rules not processed for user `df57699e3cd1494d981ddb451a56c899`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:08.248901 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-8af3abb2-8f81-463e-9d0a-eba3ca0dd3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:08.512279 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb6f42-a879-4fd8-88cd-4ae034acf793 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:08.526538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-8fcb6f42-a879-4fd8-88cd-4ae034acf793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:08.570410 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 942/1881] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:08.590318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3abb2-8f81-463e-9d0a-eba3ca0dd3e7 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:08.592366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-8af3abb2-8f81-463e-9d0a-eba3ca0dd3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:08.738816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb6f42-a879-4fd8-88cd-4ae034acf793 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:08.758367 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba28840-54dd-4c4e-a041-c27729c03ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:08.760596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba28840-54dd-4c4e-a041-c27729c03ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:08.767378 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba28840-54dd-4c4e-a041-c27729c03ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:08.768077 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba28840-54dd-4c4e-a041-c27729c03ede admin admin] PATH_INFO: `/v3/projects/19a8cc95f77c4e188bb4cd8f8e3b31c3` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:08.772677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba28840-54dd-4c4e-a041-c27729c03ede admin admin] RBAC: Authorizing `identity:delete_project(project_id=19a8cc95f77c4e188bb4cd8f8e3b31c3)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:08.776960 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba28840-54dd-4c4e-a041-c27729c03ede admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:08.790998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3abb2-8f81-463e-9d0a-eba3ca0dd3e7 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:08.805649 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a84b19-d3b4-44b4-a7de-0a5c510c7d31 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b64b4b172467692d02851b2955b81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:08.806538 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a84b19-d3b4-44b4-a7de-0a5c510c7d31 service glance] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:08.806822 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a84b19-d3b4-44b4-a7de-0a5c510c7d31 service glance] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:08.807103 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a84b19-d3b4-44b4-a7de-0a5c510c7d31 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:08.832022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a84b19-d3b4-44b4-a7de-0a5c510c7d31 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:08.844002 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a84b19-d3b4-44b4-a7de-0a5c510c7d31 service glance] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:08.861735 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 943/1882] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:13:08.863833 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281565d1-dbc7-4ba7-85e6-3582cb86af27 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:08.864192 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281565d1-dbc7-4ba7-85e6-3582cb86af27 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:08.864560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281565d1-dbc7-4ba7-85e6-3582cb86af27 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:08.866375 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 944/1883] 10.222.0.73 () {54 vars in 921 bytes} [Mon Aug 29 20:13:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:13:08.917909 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-cba28840-54dd-4c4e-a041-c27729c03ede admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19a8cc95f77c4e188bb4cd8f8e3b31c3', 'request_id': 'req-cba28840-54dd-4c4e-a041-c27729c03ede'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:08.926868 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-cba28840-54dd-4c4e-a041-c27729c03ede admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19a8cc95f77c4e188bb4cd8f8e3b31c3', 'request_id': 'req-cba28840-54dd-4c4e-a041-c27729c03ede'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:08.934847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-cba28840-54dd-4c4e-a041-c27729c03ede admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19a8cc95f77c4e188bb4cd8f8e3b31c3', 'request_id': 'req-cba28840-54dd-4c4e-a041-c27729c03ede'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:08.957142 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-cba28840-54dd-4c4e-a041-c27729c03ede admin admin] The token cache is being invalidate because project 19a8cc95f77c4e188bb4cd8f8e3b31c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:13:08.957728 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-cba28840-54dd-4c4e-a041-c27729c03ede admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:08.958757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-cba28840-54dd-4c4e-a041-c27729c03ede admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:08.960514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 940/1884] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:13:08 2022] DELETE /identity/v3/projects/19a8cc95f77c4e188bb4cd8f8e3b31c3 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:08.966563 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00217e7d-8aca-4ae6-bd72-902a4818d461 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:08.966983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00217e7d-8aca-4ae6-bd72-902a4818d461 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:08.967283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00217e7d-8aca-4ae6-bd72-902a4818d461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:09.016228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-00217e7d-8aca-4ae6-bd72-902a4818d461 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:09.018094 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-00217e7d-8aca-4ae6-bd72-902a4818d461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:09.136995 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:09.195276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 945/1885] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:09.206406 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fc0ad-2354-47f6-aae9-2741af3b713b None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:09.207087 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fc0ad-2354-47f6-aae9-2741af3b713b None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:09.207572 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fc0ad-2354-47f6-aae9-2741af3b713b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:09.289079 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-d53fc0ad-2354-47f6-aae9-2741af3b713b None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:09.296985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d53fc0ad-2354-47f6-aae9-2741af3b713b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:09.306757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 941/1886] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:09.311328 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00217e7d-8aca-4ae6-bd72-902a4818d461 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:09.333260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00217e7d-8aca-4ae6-bd72-902a4818d461 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:09.344291 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e19e1b-f8ef-4255-829c-15ad2ad83007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:09.345016 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e19e1b-f8ef-4255-829c-15ad2ad83007 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:09.345219 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e19e1b-f8ef-4255-829c-15ad2ad83007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:09.345495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e19e1b-f8ef-4255-829c-15ad2ad83007 admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:09.346152 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e19e1b-f8ef-4255-829c-15ad2ad83007 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:13:09.346460 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e19e1b-f8ef-4255-829c-15ad2ad83007 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:09.348089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e19e1b-f8ef-4255-829c-15ad2ad83007 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:09.355063 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0b32f-ab71-47ae-a84e-bbe882cab617 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:09.355395 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0b32f-ab71-47ae-a84e-bbe882cab617 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:09.355646 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0b32f-ab71-47ae-a84e-bbe882cab617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:09.391155 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-57e0b32f-ab71-47ae-a84e-bbe882cab617 None None] MFA Rules not processed for user `95cf77a92a174ff8b0503baf5c7e63be`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:09.391992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 946/1887] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:13:09.394089 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-57e0b32f-ab71-47ae-a84e-bbe882cab617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:09.396595 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e19e1b-f8ef-4255-829c-15ad2ad83007 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:09.412084 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e19e1b-f8ef-4255-829c-15ad2ad83007 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:09.433191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1af039-8ad5-4e32-b7bf-c94765c9135f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:09.434911 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1af039-8ad5-4e32-b7bf-c94765c9135f admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:09.436744 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1af039-8ad5-4e32-b7bf-c94765c9135f admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:09.437092 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1af039-8ad5-4e32-b7bf-c94765c9135f admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:09.437746 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1af039-8ad5-4e32-b7bf-c94765c9135f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:09.440385 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1af039-8ad5-4e32-b7bf-c94765c9135f admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:09.460891 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:09.613951 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 947/1888] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:13:09 2022] POST /identity/v3/projects => generated 383 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:13:09.624672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1af039-8ad5-4e32-b7bf-c94765c9135f admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:09.764572 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 942/1889] 10.222.0.73 () {58 vars in 1018 bytes} [Mon Aug 29 20:13:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 29 20:13:09.838746 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1af039-8ad5-4e32-b7bf-c94765c9135f admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:09.893368 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abef3f-c9d9-4e6d-b659-aa8c9cfe7269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:09.893368 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abef3f-c9d9-4e6d-b659-aa8c9cfe7269 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:09.893368 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abef3f-c9d9-4e6d-b659-aa8c9cfe7269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:09.893368 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abef3f-c9d9-4e6d-b659-aa8c9cfe7269 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:09.893995 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71abef3f-c9d9-4e6d-b659-aa8c9cfe7269 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:09.893995 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71abef3f-c9d9-4e6d-b659-aa8c9cfe7269 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:09.938130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0b32f-ab71-47ae-a84e-bbe882cab617 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:09.957751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 948/1890] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:13:09 2022] POST /identity/v3/users => generated 292 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:09.958217 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0b32f-ab71-47ae-a84e-bbe882cab617 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:09.980779 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617b83c-343c-4389-b43b-eef39a4065e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:09.980779 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617b83c-343c-4389-b43b-eef39a4065e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:09.980779 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617b83c-343c-4389-b43b-eef39a4065e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:09.980779 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617b83c-343c-4389-b43b-eef39a4065e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:09.982030 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617b83c-343c-4389-b43b-eef39a4065e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:09.982030 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617b83c-343c-4389-b43b-eef39a4065e9 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:09.986373 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abef3f-c9d9-4e6d-b659-aa8c9cfe7269 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:09.987764 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 943/1891] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:10.012019 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617b83c-343c-4389-b43b-eef39a4065e9 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:10.014784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abef3f-c9d9-4e6d-b659-aa8c9cfe7269 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:10.015106 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-4617b83c-343c-4389-b43b-eef39a4065e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:10.044277 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631a9d1-9bce-4f09-900f-e074b1600da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:10.049804 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631a9d1-9bce-4f09-900f-e074b1600da6 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:10.050358 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631a9d1-9bce-4f09-900f-e074b1600da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:10.053967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631a9d1-9bce-4f09-900f-e074b1600da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:10.060669 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631a9d1-9bce-4f09-900f-e074b1600da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:10.064448 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631a9d1-9bce-4f09-900f-e074b1600da6 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:10.124570 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 949/1892] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:10.128340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631a9d1-9bce-4f09-900f-e074b1600da6 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:10.170600 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631a9d1-9bce-4f09-900f-e074b1600da6 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:10.188299 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d2402-6380-4a9c-9d93-2827293f1e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:10.192542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d2402-6380-4a9c-9d93-2827293f1e72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:10.193832 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d2402-6380-4a9c-9d93-2827293f1e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:10.194425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d2402-6380-4a9c-9d93-2827293f1e72 admin admin] PATH_INFO: `/v3/projects/e0f6ef9ea60946ca8e92ed540d67de03/users/bea2f272264f4615b4234a0e371f39aa/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:10.254762 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d2402-6380-4a9c-9d93-2827293f1e72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0f6ef9ea60946ca8e92ed540d67de03, user_id=bea2f272264f4615b4234a0e371f39aa, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:10.269578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d2402-6380-4a9c-9d93-2827293f1e72 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:10.292499 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617b83c-343c-4389-b43b-eef39a4065e9 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:10.308333 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 950/1893] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:10 2022] PUT /identity/v3/projects/e0f6ef9ea60946ca8e92ed540d67de03/users/bea2f272264f4615b4234a0e371f39aa/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:10.330377 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9821a47-0af1-4dd0-85eb-3ce70a5012db None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:10.333770 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9821a47-0af1-4dd0-85eb-3ce70a5012db None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:10.336003 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9821a47-0af1-4dd0-85eb-3ce70a5012db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:10.460329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-e9821a47-0af1-4dd0-85eb-3ce70a5012db None None] MFA Rules not processed for user `bea2f272264f4615b4234a0e371f39aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:10.475166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-e9821a47-0af1-4dd0-85eb-3ce70a5012db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:10.639345 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbc822-91de-43cb-a81b-02dc9c269783 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:10.640239 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbc822-91de-43cb-a81b-02dc9c269783 service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:10.640536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbc822-91de-43cb-a81b-02dc9c269783 service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:10.640919 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbc822-91de-43cb-a81b-02dc9c269783 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:10.673911 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfbc822-91de-43cb-a81b-02dc9c269783 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:10.675687 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfbc822-91de-43cb-a81b-02dc9c269783 service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:10.701442 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 944/1894] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 690 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:13:10.740636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 951/1895] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 421 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:10.743093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbc822-91de-43cb-a81b-02dc9c269783 service nova] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:10.826353 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:10.884395 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbc822-91de-43cb-a81b-02dc9c269783 service nova] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:10.894376 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cef24-02c1-4077-b4e2-f3cb2eaf51ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:10.895665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cef24-02c1-4077-b4e2-f3cb2eaf51ee admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:10.896575 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cef24-02c1-4077-b4e2-f3cb2eaf51ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:10.896861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cef24-02c1-4077-b4e2-f3cb2eaf51ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:10.909538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cef24-02c1-4077-b4e2-f3cb2eaf51ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:10.914775 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cef24-02c1-4077-b4e2-f3cb2eaf51ee admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:10.963692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 945/1896] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:13:10 2022] POST /identity/v3/projects => generated 381 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:10.967944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9821a47-0af1-4dd0-85eb-3ce70a5012db None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:10.984041 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9821a47-0af1-4dd0-85eb-3ce70a5012db None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:10.991732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:11.023791 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cef24-02c1-4077-b4e2-f3cb2eaf51ee admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:11.026823 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-216cef24-02c1-4077-b4e2-f3cb2eaf51ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:11.130126 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb3c9a-e7e3-476d-9452-58b242eaa18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:11.130126 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb3c9a-e7e3-476d-9452-58b242eaa18b admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:11.130126 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb3c9a-e7e3-476d-9452-58b242eaa18b admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:11.130789 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb3c9a-e7e3-476d-9452-58b242eaa18b admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:11.133187 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb3c9a-e7e3-476d-9452-58b242eaa18b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:11.145284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb3c9a-e7e3-476d-9452-58b242eaa18b admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:11.209971 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 952/1897] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:13:10 2022] POST /identity/v3/users => generated 297 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:11.213183 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb3c9a-e7e3-476d-9452-58b242eaa18b admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:11.253128 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb3c9a-e7e3-476d-9452-58b242eaa18b admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:11.264933 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c2424-9ce8-4875-a6a7-b69e2d1e207a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:11.265726 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c2424-9ce8-4875-a6a7-b69e2d1e207a admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:11.266126 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c2424-9ce8-4875-a6a7-b69e2d1e207a admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:11.266461 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c2424-9ce8-4875-a6a7-b69e2d1e207a admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:11.267237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c2424-9ce8-4875-a6a7-b69e2d1e207a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:11.269181 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c2424-9ce8-4875-a6a7-b69e2d1e207a admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:11.286506 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cef24-02c1-4077-b4e2-f3cb2eaf51ee admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:11.294151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 953/1898] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:11.303773 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c2424-9ce8-4875-a6a7-b69e2d1e207a admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:11.306644 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036e73d-ec41-4204-90ef-b6c8094d07f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:11.308036 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036e73d-ec41-4204-90ef-b6c8094d07f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:11.308535 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036e73d-ec41-4204-90ef-b6c8094d07f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:11.308884 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036e73d-ec41-4204-90ef-b6c8094d07f5 admin admin] PATH_INFO: `/v3/projects/453aa10dbf2d4aeb81c81a8169a8bf3e` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:11.310507 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036e73d-ec41-4204-90ef-b6c8094d07f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=453aa10dbf2d4aeb81c81a8169a8bf3e)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:11.314331 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036e73d-ec41-4204-90ef-b6c8094d07f5 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:11.322670 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c2424-9ce8-4875-a6a7-b69e2d1e207a admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:11.331747 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90098066-ea5a-414f-8085-bd0c7ee60d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:11.332831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90098066-ea5a-414f-8085-bd0c7ee60d9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:11.333300 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90098066-ea5a-414f-8085-bd0c7ee60d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:11.334750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90098066-ea5a-414f-8085-bd0c7ee60d9c admin admin] PATH_INFO: `/v3/projects/f8007f894c0d4093ba4d00333206ab8f/users/a2f60bceaec84009a9b40c35b40d40b6/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:11.360727 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90098066-ea5a-414f-8085-bd0c7ee60d9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8007f894c0d4093ba4d00333206ab8f, user_id=a2f60bceaec84009a9b40c35b40d40b6, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:11.363677 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90098066-ea5a-414f-8085-bd0c7ee60d9c admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:11.379859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 954/1899] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:11 2022] PUT /identity/v3/projects/f8007f894c0d4093ba4d00333206ab8f/users/a2f60bceaec84009a9b40c35b40d40b6/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:11.383861 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90098066-ea5a-414f-8085-bd0c7ee60d9c admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:11.412164 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-6036e73d-ec41-4204-90ef-b6c8094d07f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '453aa10dbf2d4aeb81c81a8169a8bf3e', 'request_id': 'req-6036e73d-ec41-4204-90ef-b6c8094d07f5'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:11.420330 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-6036e73d-ec41-4204-90ef-b6c8094d07f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '453aa10dbf2d4aeb81c81a8169a8bf3e', 'request_id': 'req-6036e73d-ec41-4204-90ef-b6c8094d07f5'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:11.434467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-6036e73d-ec41-4204-90ef-b6c8094d07f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '453aa10dbf2d4aeb81c81a8169a8bf3e', 'request_id': 'req-6036e73d-ec41-4204-90ef-b6c8094d07f5'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:11.459339 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-6036e73d-ec41-4204-90ef-b6c8094d07f5 admin admin] The token cache is being invalidate because project 453aa10dbf2d4aeb81c81a8169a8bf3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:13:11.459854 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-6036e73d-ec41-4204-90ef-b6c8094d07f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:11.461319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-6036e73d-ec41-4204-90ef-b6c8094d07f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:11.464181 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 946/1900] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:13:11 2022] DELETE /identity/v3/projects/453aa10dbf2d4aeb81c81a8169a8bf3e => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:13:11.470479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689d067-bca1-4ee2-b09c-1262b6eabdfc None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:11.470975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689d067-bca1-4ee2-b09c-1262b6eabdfc None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:11.471412 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689d067-bca1-4ee2-b09c-1262b6eabdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:11.491209 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90098066-ea5a-414f-8085-bd0c7ee60d9c admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:11.508744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-b689d067-bca1-4ee2-b09c-1262b6eabdfc None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:11.511010 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-b689d067-bca1-4ee2-b09c-1262b6eabdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:11.580506 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:11.616428 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256f291-3ac2-44a0-b913-1d4778f6ffa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:11.617253 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256f291-3ac2-44a0-b913-1d4778f6ffa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:11.617541 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256f291-3ac2-44a0-b913-1d4778f6ffa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:11.618205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256f291-3ac2-44a0-b913-1d4778f6ffa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:11.619551 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256f291-3ac2-44a0-b913-1d4778f6ffa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:11.624475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256f291-3ac2-44a0-b913-1d4778f6ffa6 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:11.640811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 955/1901] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:13:11.642198 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 947/1902] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:13:11.655180 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256f291-3ac2-44a0-b913-1d4778f6ffa6 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:11.658892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305ab5f-04a7-454d-9012-8c78b9e6b408 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:11.658892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305ab5f-04a7-454d-9012-8c78b9e6b408 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:11.658892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305ab5f-04a7-454d-9012-8c78b9e6b408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:11.659266 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-e256f291-3ac2-44a0-b913-1d4778f6ffa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:11.688844 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256f291-3ac2-44a0-b913-1d4778f6ffa6 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:11.710593 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-2305ab5f-04a7-454d-9012-8c78b9e6b408 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:11.712020 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-2305ab5f-04a7-454d-9012-8c78b9e6b408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:11.712810 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4d9e3-777c-458f-80de-d0be9e9eae3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:11.713577 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4d9e3-777c-458f-80de-d0be9e9eae3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:11.713894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4d9e3-777c-458f-80de-d0be9e9eae3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:11.714174 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4d9e3-777c-458f-80de-d0be9e9eae3b admin admin] PATH_INFO: `/v3/projects/f8007f894c0d4093ba4d00333206ab8f/users/a2f60bceaec84009a9b40c35b40d40b6/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:11.721537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4d9e3-777c-458f-80de-d0be9e9eae3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8007f894c0d4093ba4d00333206ab8f, user_id=a2f60bceaec84009a9b40c35b40d40b6, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:11.732559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4d9e3-777c-458f-80de-d0be9e9eae3b admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:11.738008 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:11.751760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 948/1903] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:11.754522 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2305ab5f-04a7-454d-9012-8c78b9e6b408 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:11.777768 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 956/1904] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:11 2022] PUT /identity/v3/projects/f8007f894c0d4093ba4d00333206ab8f/users/a2f60bceaec84009a9b40c35b40d40b6/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:11.778379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2305ab5f-04a7-454d-9012-8c78b9e6b408 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:11.782186 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0158541-11bd-4383-a14f-6139fe4ccbcb None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:11.783113 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0158541-11bd-4383-a14f-6139fe4ccbcb None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:11.783449 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0158541-11bd-4383-a14f-6139fe4ccbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:11.847387 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-e0158541-11bd-4383-a14f-6139fe4ccbcb None None] MFA Rules not processed for user `a2f60bceaec84009a9b40c35b40d40b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:11.848689 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-e0158541-11bd-4383-a14f-6139fe4ccbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:11.908640 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcc827-e0da-4a67-949d-53c2a86c1295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:11.909323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcc827-e0da-4a67-949d-53c2a86c1295 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:11.909595 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcc827-e0da-4a67-949d-53c2a86c1295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:11.909848 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcc827-e0da-4a67-949d-53c2a86c1295 admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:11.910506 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbcc827-e0da-4a67-949d-53c2a86c1295 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:13:11.910803 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbcc827-e0da-4a67-949d-53c2a86c1295 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:11.912700 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbcc827-e0da-4a67-949d-53c2a86c1295 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:11.933512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 949/1905] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:13:11.939512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcc827-e0da-4a67-949d-53c2a86c1295 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:11.958360 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcc827-e0da-4a67-949d-53c2a86c1295 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:11.969399 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 957/1906] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:11 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:13:11.970527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cee433-71a3-484a-8376-abc2f22f93c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:11.971546 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cee433-71a3-484a-8376-abc2f22f93c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:11.972217 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cee433-71a3-484a-8376-abc2f22f93c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:11.972757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cee433-71a3-484a-8376-abc2f22f93c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:11.973681 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cee433-71a3-484a-8376-abc2f22f93c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:11.982393 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cee433-71a3-484a-8376-abc2f22f93c4 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:11.983913 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0158541-11bd-4383-a14f-6139fe4ccbcb None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:11.991044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-e0158541-11bd-4383-a14f-6139fe4ccbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:12.053750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 950/1907] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:13:11 2022] POST /identity/v3/projects => generated 407 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:12.058928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cee433-71a3-484a-8376-abc2f22f93c4 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:12.155558 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0158541-11bd-4383-a14f-6139fe4ccbcb None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:12.237603 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cee433-71a3-484a-8376-abc2f22f93c4 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:12.254136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a771be-05d3-44a5-9d13-d58fd0c9095e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:12.254136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a771be-05d3-44a5-9d13-d58fd0c9095e admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:12.254136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a771be-05d3-44a5-9d13-d58fd0c9095e admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:12.254136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a771be-05d3-44a5-9d13-d58fd0c9095e admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:12.254598 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a771be-05d3-44a5-9d13-d58fd0c9095e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:12.254598 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a771be-05d3-44a5-9d13-d58fd0c9095e admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:12.262557 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c64e1-bd93-4dac-be83-0afc844a9c56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:12.263422 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c64e1-bd93-4dac-be83-0afc844a9c56 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:12.263773 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c64e1-bd93-4dac-be83-0afc844a9c56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:12.264159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c64e1-bd93-4dac-be83-0afc844a9c56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:12.296405 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 951/1908] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:13:12 2022] POST /identity/v3/users => generated 304 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:12.298390 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a771be-05d3-44a5-9d13-d58fd0c9095e admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:12.325211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a771be-05d3-44a5-9d13-d58fd0c9095e admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:12.338112 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037491b-2662-464d-93d7-19110782dd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:12.338112 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037491b-2662-464d-93d7-19110782dd7c admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:12.338112 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037491b-2662-464d-93d7-19110782dd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:12.338112 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037491b-2662-464d-93d7-19110782dd7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:12.338497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7037491b-2662-464d-93d7-19110782dd7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:12.338497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7037491b-2662-464d-93d7-19110782dd7c admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:12.351695 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 952/1909] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:13:12.361799 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037491b-2662-464d-93d7-19110782dd7c admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:12.364551 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:12.374622 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037491b-2662-464d-93d7-19110782dd7c admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:12.381863 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82833fb-f5a5-4424-9ab2-93a19b37e19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:12.382506 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82833fb-f5a5-4424-9ab2-93a19b37e19e admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:12.382786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82833fb-f5a5-4424-9ab2-93a19b37e19e admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:12.383065 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82833fb-f5a5-4424-9ab2-93a19b37e19e admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:12.383736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82833fb-f5a5-4424-9ab2-93a19b37e19e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:12.385358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82833fb-f5a5-4424-9ab2-93a19b37e19e admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:12.391564 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0c64e1-bd93-4dac-be83-0afc844a9c56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:12.393634 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0c64e1-bd93-4dac-be83-0afc844a9c56 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:12.412748 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 953/1910] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:13:12.417279 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82833fb-f5a5-4424-9ab2-93a19b37e19e admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:12.418419 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 958/1911] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:11 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:13:12.431391 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82833fb-f5a5-4424-9ab2-93a19b37e19e admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:12.439344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88115d2-c7f6-4eb3-a92f-c4b54392eaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:12.440066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88115d2-c7f6-4eb3-a92f-c4b54392eaae admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:12.440348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88115d2-c7f6-4eb3-a92f-c4b54392eaae admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:12.440787 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88115d2-c7f6-4eb3-a92f-c4b54392eaae admin admin] PATH_INFO: `/v3/projects/b1aae31d82a94679acd5afb18799eb9d/users/aa269bcb3e3e4f8a868856f4850f9a25/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:12.462382 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88115d2-c7f6-4eb3-a92f-c4b54392eaae admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1aae31d82a94679acd5afb18799eb9d, user_id=aa269bcb3e3e4f8a868856f4850f9a25, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:12.465822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88115d2-c7f6-4eb3-a92f-c4b54392eaae admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:12.470349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:12.498751 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 954/1912] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:12 2022] PUT /identity/v3/projects/b1aae31d82a94679acd5afb18799eb9d/users/aa269bcb3e3e4f8a868856f4850f9a25/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:13:12.501080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2da09-80fd-4073-a254-b423e20b156b None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:12.501966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2da09-80fd-4073-a254-b423e20b156b None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:12.502317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2da09-80fd-4073-a254-b423e20b156b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:12.546235 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-a8e2da09-80fd-4073-a254-b423e20b156b None None] MFA Rules not processed for user `aa269bcb3e3e4f8a868856f4850f9a25`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:12.547924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a8e2da09-80fd-4073-a254-b423e20b156b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:12.573154 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88115d2-c7f6-4eb3-a92f-c4b54392eaae admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:12.655205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 959/1913] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:12 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:12.660435 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2da09-80fd-4073-a254-b423e20b156b None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:12.662002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88115d2-c7f6-4eb3-a92f-c4b54392eaae admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:12.673117 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d22ec-62ac-48d4-9347-6a8c0bb178c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:12.673117 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d22ec-62ac-48d4-9347-6a8c0bb178c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:12.673117 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d22ec-62ac-48d4-9347-6a8c0bb178c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:12.673117 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d22ec-62ac-48d4-9347-6a8c0bb178c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:12.673575 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-973d22ec-62ac-48d4-9347-6a8c0bb178c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:12.740227 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2da09-80fd-4073-a254-b423e20b156b None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:12.746814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b3af2-a576-404c-86d3-fa9eded1f0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:12.747412 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b3af2-a576-404c-86d3-fa9eded1f0c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:12.747869 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b3af2-a576-404c-86d3-fa9eded1f0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:12.748145 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b3af2-a576-404c-86d3-fa9eded1f0c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:12.748830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b3af2-a576-404c-86d3-fa9eded1f0c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:12.750964 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b3af2-a576-404c-86d3-fa9eded1f0c5 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:12.751543 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d22ec-62ac-48d4-9347-6a8c0bb178c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:12.753692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d22ec-62ac-48d4-9347-6a8c0bb178c5 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:12.769083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 955/1914] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:13:12.795788 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 960/1915] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:13:12 2022] POST /identity/v3/projects => generated 409 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:13:12.801430 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d22ec-62ac-48d4-9347-6a8c0bb178c5 service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:12.817909 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d22ec-62ac-48d4-9347-6a8c0bb178c5 service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:12.906002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228b9d9-e3a0-49ee-90c8-42bea000fca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:12.907298 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228b9d9-e3a0-49ee-90c8-42bea000fca2 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:12.907584 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228b9d9-e3a0-49ee-90c8-42bea000fca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:12.907867 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228b9d9-e3a0-49ee-90c8-42bea000fca2 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:12.908531 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a228b9d9-e3a0-49ee-90c8-42bea000fca2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:12.912034 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a228b9d9-e3a0-49ee-90c8-42bea000fca2 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:12.952809 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 956/1916] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:13:12 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:12.955912 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b3af2-a576-404c-86d3-fa9eded1f0c5 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:12.979765 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b3af2-a576-404c-86d3-fa9eded1f0c5 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:12.993412 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261fe59-e670-42ad-9ae6-f59ebef3510b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:12.994420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261fe59-e670-42ad-9ae6-f59ebef3510b admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:12.995283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261fe59-e670-42ad-9ae6-f59ebef3510b admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:12.995478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261fe59-e670-42ad-9ae6-f59ebef3510b admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:12.996553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261fe59-e670-42ad-9ae6-f59ebef3510b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:12.999901 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261fe59-e670-42ad-9ae6-f59ebef3510b admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:13.021310 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 961/1917] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:13.027274 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228b9d9-e3a0-49ee-90c8-42bea000fca2 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:13.043587 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228b9d9-e3a0-49ee-90c8-42bea000fca2 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:13.052432 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa36ac2-3a94-41f7-a1df-227caae6b08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:13.054083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa36ac2-3a94-41f7-a1df-227caae6b08e admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:13.054648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa36ac2-3a94-41f7-a1df-227caae6b08e admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:13.055346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa36ac2-3a94-41f7-a1df-227caae6b08e admin admin] PATH_INFO: `/v3/projects/c3c7ba4578c54018962163856667f3f4/users/28cf2abf2d3f42b9afea0e121ed1ad25/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:13.109895 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa36ac2-3a94-41f7-a1df-227caae6b08e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3c7ba4578c54018962163856667f3f4, user_id=28cf2abf2d3f42b9afea0e121ed1ad25, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:13.115138 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa36ac2-3a94-41f7-a1df-227caae6b08e admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:13.159848 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 957/1918] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:13 2022] PUT /identity/v3/projects/c3c7ba4578c54018962163856667f3f4/users/28cf2abf2d3f42b9afea0e121ed1ad25/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:13.164456 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261fe59-e670-42ad-9ae6-f59ebef3510b admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:13.180345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261fe59-e670-42ad-9ae6-f59ebef3510b admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:13.339605 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:13.387077 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c2ea13-131c-4291-a243-788f0e47fe5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:13.390747 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c2ea13-131c-4291-a243-788f0e47fe5a admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:13.390747 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c2ea13-131c-4291-a243-788f0e47fe5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:13.391645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c2ea13-131c-4291-a243-788f0e47fe5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:13.392305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c2ea13-131c-4291-a243-788f0e47fe5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:13.397668 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c2ea13-131c-4291-a243-788f0e47fe5a admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:13.442355 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa36ac2-3a94-41f7-a1df-227caae6b08e admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:13.449489 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 962/1919] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:13.455578 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa36ac2-3a94-41f7-a1df-227caae6b08e admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:13.467516 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f722272-5444-44cf-82e6-c6c8cf903ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:13.468208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f722272-5444-44cf-82e6-c6c8cf903ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:13.468517 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f722272-5444-44cf-82e6-c6c8cf903ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:13.468809 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f722272-5444-44cf-82e6-c6c8cf903ea3 admin admin] PATH_INFO: `/v3/projects/c3c7ba4578c54018962163856667f3f4/users/28cf2abf2d3f42b9afea0e121ed1ad25/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:13.476983 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f722272-5444-44cf-82e6-c6c8cf903ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3c7ba4578c54018962163856667f3f4, user_id=28cf2abf2d3f42b9afea0e121ed1ad25, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:13.482497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f722272-5444-44cf-82e6-c6c8cf903ea3 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:13.498041 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:13.507954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 958/1920] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:13 2022] PUT /identity/v3/projects/c3c7ba4578c54018962163856667f3f4/users/28cf2abf2d3f42b9afea0e121ed1ad25/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:13.520424 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe38d80-ed44-4440-9c5b-98e56b506560 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:13.520977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe38d80-ed44-4440-9c5b-98e56b506560 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:13.521259 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe38d80-ed44-4440-9c5b-98e56b506560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:13.565882 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-bbe38d80-ed44-4440-9c5b-98e56b506560 None None] MFA Rules not processed for user `28cf2abf2d3f42b9afea0e121ed1ad25`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:13.567404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-bbe38d80-ed44-4440-9c5b-98e56b506560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:13.707523 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 963/1921] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:13 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:13.721216 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f722272-5444-44cf-82e6-c6c8cf903ea3 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:13.827778 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f722272-5444-44cf-82e6-c6c8cf903ea3 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:13.833986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe97cbd-ff2b-49b6-b4d3-a98cef3b73ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:13.834626 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe97cbd-ff2b-49b6-b4d3-a98cef3b73ba service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:13.834903 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe97cbd-ff2b-49b6-b4d3-a98cef3b73ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:13.835180 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe97cbd-ff2b-49b6-b4d3-a98cef3b73ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:13.848764 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe97cbd-ff2b-49b6-b4d3-a98cef3b73ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:13.849592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe97cbd-ff2b-49b6-b4d3-a98cef3b73ba service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:13.863284 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 959/1922] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:13 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:13:14.071653 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe38d80-ed44-4440-9c5b-98e56b506560 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:14.087088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe38d80-ed44-4440-9c5b-98e56b506560 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:14.091112 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b4639-1025-40ca-b1b7-7b501383d555 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:14.097412 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b4639-1025-40ca-b1b7-7b501383d555 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:14.097745 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b4639-1025-40ca-b1b7-7b501383d555 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:14.097992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b4639-1025-40ca-b1b7-7b501383d555 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:14.199368 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b4639-1025-40ca-b1b7-7b501383d555 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:14.202967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b4639-1025-40ca-b1b7-7b501383d555 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:14.217389 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 964/1923] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:14 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:13:18.698872 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8602ba-3337-4a6b-a6c5-c7767c39a92d None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:18.699395 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8602ba-3337-4a6b-a6c5-c7767c39a92d None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:18.700052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8602ba-3337-4a6b-a6c5-c7767c39a92d None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:18.702030 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 960/1924] 10.222.0.73 () {54 vars in 921 bytes} [Mon Aug 29 20:13:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:13:18.714647 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb8448-f57f-4886-915e-9b464346e770 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:18.715150 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb8448-f57f-4886-915e-9b464346e770 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:18.715236 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb8448-f57f-4886-915e-9b464346e770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:18.740245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-d5bb8448-f57f-4886-915e-9b464346e770 None None] MFA Rules not processed for user `95cf77a92a174ff8b0503baf5c7e63be`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:18.741501 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-d5bb8448-f57f-4886-915e-9b464346e770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:18.837728 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:18.849509 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 965/1925] 10.222.0.73 () {58 vars in 1018 bytes} [Mon Aug 29 20:13:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:18.862564 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8602ba-3337-4a6b-a6c5-c7767c39a92d None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:18.864325 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-5b8602ba-3337-4a6b-a6c5-c7767c39a92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:18.884401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8602ba-3337-4a6b-a6c5-c7767c39a92d None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:18.891740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0599c-37fe-4156-9b61-ec9bd83385a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:18.892378 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0599c-37fe-4156-9b61-ec9bd83385a5 service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:18.892665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0599c-37fe-4156-9b61-ec9bd83385a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:18.892944 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0599c-37fe-4156-9b61-ec9bd83385a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:18.903154 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0599c-37fe-4156-9b61-ec9bd83385a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:18.905414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0599c-37fe-4156-9b61-ec9bd83385a5 service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:18.917394 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 961/1926] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:13:25.997264 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb8448-f57f-4886-915e-9b464346e770 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:26.173807 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb8448-f57f-4886-915e-9b464346e770 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:26.179220 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d83df5-3416-4e7f-85a5-c5b7e07e65eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:26.179882 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d83df5-3416-4e7f-85a5-c5b7e07e65eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:26.180161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d83df5-3416-4e7f-85a5-c5b7e07e65eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:26.180501 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d83df5-3416-4e7f-85a5-c5b7e07e65eb admin admin] PATH_INFO: `/v3/users/bea2f272264f4615b4234a0e371f39aa` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:26.181683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d83df5-3416-4e7f-85a5-c5b7e07e65eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bea2f272264f4615b4234a0e371f39aa)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:26.185259 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d83df5-3416-4e7f-85a5-c5b7e07e65eb admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:26.317156 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e1d83df5-3416-4e7f-85a5-c5b7e07e65eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bea2f272264f4615b4234a0e371f39aa'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:26.367966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e1d83df5-3416-4e7f-85a5-c5b7e07e65eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bea2f272264f4615b4234a0e371f39aa'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:26.400890 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e1d83df5-3416-4e7f-85a5-c5b7e07e65eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bea2f272264f4615b4234a0e371f39aa'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:26.414077 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e1d83df5-3416-4e7f-85a5-c5b7e07e65eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bea2f272264f4615b4234a0e371f39aa'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:26.416414 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e1d83df5-3416-4e7f-85a5-c5b7e07e65eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bea2f272264f4615b4234a0e371f39aa'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:26.428635 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 966/1927] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:13:25 2022] DELETE /identity/v3/users/bea2f272264f4615b4234a0e371f39aa => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:26.437326 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fda25-783a-43c8-84c1-88c27e705c73 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:26.438805 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fda25-783a-43c8-84c1-88c27e705c73 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:26.443909 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fda25-783a-43c8-84c1-88c27e705c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:26.494002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-a89fda25-783a-43c8-84c1-88c27e705c73 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:26.498053 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-a89fda25-783a-43c8-84c1-88c27e705c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:26.667375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 962/1928] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:26.723571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d83df5-3416-4e7f-85a5-c5b7e07e65eb admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:26.737647 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-e1d83df5-3416-4e7f-85a5-c5b7e07e65eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:26.911142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d83df5-3416-4e7f-85a5-c5b7e07e65eb admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:26.920657 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce32f35-b35c-4917-907b-1e498dd80831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:26.921310 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce32f35-b35c-4917-907b-1e498dd80831 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:26.921594 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce32f35-b35c-4917-907b-1e498dd80831 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:26.921910 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce32f35-b35c-4917-907b-1e498dd80831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:26.936214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce32f35-b35c-4917-907b-1e498dd80831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:26.937617 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce32f35-b35c-4917-907b-1e498dd80831 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:26.989987 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 967/1929] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:13:27.786903 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fda25-783a-43c8-84c1-88c27e705c73 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:27.788512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-a89fda25-783a-43c8-84c1-88c27e705c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:27.823466 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fda25-783a-43c8-84c1-88c27e705c73 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:27.837901 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a52f1-85d4-4fa4-8c97-7f9e7bf58f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:27.842365 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a52f1-85d4-4fa4-8c97-7f9e7bf58f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:27.842915 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a52f1-85d4-4fa4-8c97-7f9e7bf58f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:27.844325 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a52f1-85d4-4fa4-8c97-7f9e7bf58f16 admin admin] PATH_INFO: `/v3/projects/e0f6ef9ea60946ca8e92ed540d67de03` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:27.852336 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a52f1-85d4-4fa4-8c97-7f9e7bf58f16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0f6ef9ea60946ca8e92ed540d67de03)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:27.857226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a52f1-85d4-4fa4-8c97-7f9e7bf58f16 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:27.974420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d38a52f1-85d4-4fa4-8c97-7f9e7bf58f16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0f6ef9ea60946ca8e92ed540d67de03', 'request_id': 'req-d38a52f1-85d4-4fa4-8c97-7f9e7bf58f16'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:27.988248 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d38a52f1-85d4-4fa4-8c97-7f9e7bf58f16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0f6ef9ea60946ca8e92ed540d67de03', 'request_id': 'req-d38a52f1-85d4-4fa4-8c97-7f9e7bf58f16'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:27.999678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d38a52f1-85d4-4fa4-8c97-7f9e7bf58f16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0f6ef9ea60946ca8e92ed540d67de03', 'request_id': 'req-d38a52f1-85d4-4fa4-8c97-7f9e7bf58f16'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:28.034270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d38a52f1-85d4-4fa4-8c97-7f9e7bf58f16 admin admin] The token cache is being invalidate because project e0f6ef9ea60946ca8e92ed540d67de03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:13:28.034607 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d38a52f1-85d4-4fa4-8c97-7f9e7bf58f16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:28.035506 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d38a52f1-85d4-4fa4-8c97-7f9e7bf58f16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:28.040918 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 963/1930] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:13:27 2022] DELETE /identity/v3/projects/e0f6ef9ea60946ca8e92ed540d67de03 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:28.044395 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce32f35-b35c-4917-907b-1e498dd80831 service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:28.062039 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce32f35-b35c-4917-907b-1e498dd80831 service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:28.132710 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:28.284025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6d177-1edf-40fa-b177-1d9952193881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:28.286022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6d177-1edf-40fa-b177-1d9952193881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:28.286815 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6d177-1edf-40fa-b177-1d9952193881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:28.287381 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6d177-1edf-40fa-b177-1d9952193881 admin admin] PATH_INFO: `/v3/users/a2f60bceaec84009a9b40c35b40d40b6` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:28.289555 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6d177-1edf-40fa-b177-1d9952193881 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2f60bceaec84009a9b40c35b40d40b6)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:28.293457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6d177-1edf-40fa-b177-1d9952193881 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:28.436137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-dec6d177-1edf-40fa-b177-1d9952193881 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2f60bceaec84009a9b40c35b40d40b6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:28.472831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-dec6d177-1edf-40fa-b177-1d9952193881 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2f60bceaec84009a9b40c35b40d40b6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:28.524972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-dec6d177-1edf-40fa-b177-1d9952193881 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2f60bceaec84009a9b40c35b40d40b6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:28.547468 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-dec6d177-1edf-40fa-b177-1d9952193881 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2f60bceaec84009a9b40c35b40d40b6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:28.550187 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-dec6d177-1edf-40fa-b177-1d9952193881 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2f60bceaec84009a9b40c35b40d40b6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:28.558317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 968/1931] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:13:28 2022] DELETE /identity/v3/users/a2f60bceaec84009a9b40c35b40d40b6 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:29.093779 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a52f1-85d4-4fa4-8c97-7f9e7bf58f16 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:29.095976 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d38a52f1-85d4-4fa4-8c97-7f9e7bf58f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:29.118729 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a52f1-85d4-4fa4-8c97-7f9e7bf58f16 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:29.132897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:29.212141 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84299d5f-4d81-4a6f-96a0-5f296ec6b707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:29.212999 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84299d5f-4d81-4a6f-96a0-5f296ec6b707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:29.213338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84299d5f-4d81-4a6f-96a0-5f296ec6b707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:29.213813 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84299d5f-4d81-4a6f-96a0-5f296ec6b707 admin admin] PATH_INFO: `/v3/projects/f8007f894c0d4093ba4d00333206ab8f` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:29.215246 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84299d5f-4d81-4a6f-96a0-5f296ec6b707 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8007f894c0d4093ba4d00333206ab8f)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:29.217953 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84299d5f-4d81-4a6f-96a0-5f296ec6b707 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:29.303186 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-84299d5f-4d81-4a6f-96a0-5f296ec6b707 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8007f894c0d4093ba4d00333206ab8f', 'request_id': 'req-84299d5f-4d81-4a6f-96a0-5f296ec6b707'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:29.316897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-84299d5f-4d81-4a6f-96a0-5f296ec6b707 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8007f894c0d4093ba4d00333206ab8f', 'request_id': 'req-84299d5f-4d81-4a6f-96a0-5f296ec6b707'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:29.325408 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-84299d5f-4d81-4a6f-96a0-5f296ec6b707 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8007f894c0d4093ba4d00333206ab8f', 'request_id': 'req-84299d5f-4d81-4a6f-96a0-5f296ec6b707'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:29.343389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-84299d5f-4d81-4a6f-96a0-5f296ec6b707 admin admin] The token cache is being invalidate because project f8007f894c0d4093ba4d00333206ab8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:13:29.343893 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-84299d5f-4d81-4a6f-96a0-5f296ec6b707 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:29.344800 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-84299d5f-4d81-4a6f-96a0-5f296ec6b707 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:29.346845 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 964/1932] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:13:29 2022] DELETE /identity/v3/projects/f8007f894c0d4093ba4d00333206ab8f => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:13:29.352224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294feec0-1e3a-40dc-8fe4-1830c7c5c7b5 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:29.353067 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294feec0-1e3a-40dc-8fe4-1830c7c5c7b5 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:29.353236 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294feec0-1e3a-40dc-8fe4-1830c7c5c7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:29.378632 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-294feec0-1e3a-40dc-8fe4-1830c7c5c7b5 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:29.379906 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-294feec0-1e3a-40dc-8fe4-1830c7c5c7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:29.486337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 969/1933] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:29.498916 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5fd42-5cec-4597-aef8-ec2e14d6841b None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:29.499280 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5fd42-5cec-4597-aef8-ec2e14d6841b None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:29.499757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5fd42-5cec-4597-aef8-ec2e14d6841b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:29.539792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-60f5fd42-5cec-4597-aef8-ec2e14d6841b None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:29.542085 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-60f5fd42-5cec-4597-aef8-ec2e14d6841b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:29.584196 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 965/1934] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:29.587383 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294feec0-1e3a-40dc-8fe4-1830c7c5c7b5 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:29.635463 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294feec0-1e3a-40dc-8fe4-1830c7c5c7b5 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:29.668458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf06758-0729-4ab7-8787-1d9ea48294f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:29.674081 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf06758-0729-4ab7-8787-1d9ea48294f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:29.674919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf06758-0729-4ab7-8787-1d9ea48294f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:29.676783 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf06758-0729-4ab7-8787-1d9ea48294f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:29.678725 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf06758-0729-4ab7-8787-1d9ea48294f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:13:29.679534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf06758-0729-4ab7-8787-1d9ea48294f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:29.682972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf06758-0729-4ab7-8787-1d9ea48294f4 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:29.721215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 970/1935] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:13:29.724658 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5fd42-5cec-4597-aef8-ec2e14d6841b None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:29.754225 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5fd42-5cec-4597-aef8-ec2e14d6841b None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:29.764710 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b76a6-9d33-4141-bed2-b996a679201e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:29.766153 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b76a6-9d33-4141-bed2-b996a679201e admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:29.766865 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b76a6-9d33-4141-bed2-b996a679201e admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:29.767557 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b76a6-9d33-4141-bed2-b996a679201e admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:29.768926 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b76a6-9d33-4141-bed2-b996a679201e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:29.784937 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b76a6-9d33-4141-bed2-b996a679201e admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:29.869790 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 966/1936] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:13:29 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:29.870621 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf06758-0729-4ab7-8787-1d9ea48294f4 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:29.888847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf06758-0729-4ab7-8787-1d9ea48294f4 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:29.972899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:30.033542 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b76a6-9d33-4141-bed2-b996a679201e admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:30.035454 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c93b76a6-9d33-4141-bed2-b996a679201e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:30.047540 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57216a-53f1-45f8-9817-f8e6dd10e918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:30.048721 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57216a-53f1-45f8-9817-f8e6dd10e918 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:30.049191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57216a-53f1-45f8-9817-f8e6dd10e918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:30.049869 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57216a-53f1-45f8-9817-f8e6dd10e918 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:30.050063 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b76a6-9d33-4141-bed2-b996a679201e admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:30.051656 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a57216a-53f1-45f8-9817-f8e6dd10e918 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:30.055225 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ec301-26c5-446b-9673-c98244d17712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:30.055885 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ec301-26c5-446b-9673-c98244d17712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:30.056168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ec301-26c5-446b-9673-c98244d17712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:30.056444 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ec301-26c5-446b-9673-c98244d17712 admin admin] PATH_INFO: `/v3/users/aa269bcb3e3e4f8a868856f4850f9a25` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:30.057406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a57216a-53f1-45f8-9817-f8e6dd10e918 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:30.057669 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ec301-26c5-446b-9673-c98244d17712 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa269bcb3e3e4f8a868856f4850f9a25)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:30.060099 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ec301-26c5-446b-9673-c98244d17712 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:30.137643 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 971/1937] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:13:29 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:30.144783 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57216a-53f1-45f8-9817-f8e6dd10e918 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:30.147388 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-dc2ec301-26c5-446b-9673-c98244d17712 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa269bcb3e3e4f8a868856f4850f9a25'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:30.168430 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-dc2ec301-26c5-446b-9673-c98244d17712 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa269bcb3e3e4f8a868856f4850f9a25'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:30.184435 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57216a-53f1-45f8-9817-f8e6dd10e918 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:30.206119 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76214cf9-3a9b-49ad-a4c8-a7877de3bf36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:30.211016 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76214cf9-3a9b-49ad-a4c8-a7877de3bf36 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:30.211458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76214cf9-3a9b-49ad-a4c8-a7877de3bf36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:30.211776 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76214cf9-3a9b-49ad-a4c8-a7877de3bf36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:30.213850 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76214cf9-3a9b-49ad-a4c8-a7877de3bf36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:30.216855 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:30.222441 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76214cf9-3a9b-49ad-a4c8-a7877de3bf36 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:30.232719 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-dc2ec301-26c5-446b-9673-c98244d17712 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa269bcb3e3e4f8a868856f4850f9a25'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:30.255191 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-dc2ec301-26c5-446b-9673-c98244d17712 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa269bcb3e3e4f8a868856f4850f9a25'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:30.257460 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-dc2ec301-26c5-446b-9673-c98244d17712 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa269bcb3e3e4f8a868856f4850f9a25'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:30.267079 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 972/1938] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:30.268005 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76214cf9-3a9b-49ad-a4c8-a7877de3bf36 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:30.270770 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-76214cf9-3a9b-49ad-a4c8-a7877de3bf36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:30.286457 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 967/1939] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:13:30 2022] DELETE /identity/v3/users/aa269bcb3e3e4f8a868856f4850f9a25 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:30.296714 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80079f5c-d5f4-44f7-bf53-45cc72defe9a None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:30.297170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80079f5c-d5f4-44f7-bf53-45cc72defe9a None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:30.297648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80079f5c-d5f4-44f7-bf53-45cc72defe9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:30.299839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76214cf9-3a9b-49ad-a4c8-a7877de3bf36 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:30.337581 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-80079f5c-d5f4-44f7-bf53-45cc72defe9a None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:30.339791 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-80079f5c-d5f4-44f7-bf53-45cc72defe9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:30.508088 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 968/1940] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:30.524139 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80079f5c-d5f4-44f7-bf53-45cc72defe9a None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:30.527013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-80079f5c-d5f4-44f7-bf53-45cc72defe9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:30.528901 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83c19a-6733-4268-bfcc-bd7df92a8cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:30.529659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83c19a-6733-4268-bfcc-bd7df92a8cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:30.529919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83c19a-6733-4268-bfcc-bd7df92a8cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:30.530193 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83c19a-6733-4268-bfcc-bd7df92a8cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:30.530914 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83c19a-6733-4268-bfcc-bd7df92a8cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:30.532558 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83c19a-6733-4268-bfcc-bd7df92a8cf0 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:30.550378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 973/1941] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:30.554971 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83c19a-6733-4268-bfcc-bd7df92a8cf0 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:30.568039 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83c19a-6733-4268-bfcc-bd7df92a8cf0 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:30.581396 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee338e-b804-40fb-84a4-c498ce6478ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:30.582247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee338e-b804-40fb-84a4-c498ce6478ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:30.583807 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee338e-b804-40fb-84a4-c498ce6478ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:30.584122 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee338e-b804-40fb-84a4-c498ce6478ec admin admin] PATH_INFO: `/v3/projects/92c2b524a96e42df9a48390eea768383/users/49cf3c2240bf4d3482b6c031ada51ffe/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:30.659356 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee338e-b804-40fb-84a4-c498ce6478ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=92c2b524a96e42df9a48390eea768383, user_id=49cf3c2240bf4d3482b6c031ada51ffe, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:30.666209 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee338e-b804-40fb-84a4-c498ce6478ec admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:30.725060 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 974/1942] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:30 2022] PUT /identity/v3/projects/92c2b524a96e42df9a48390eea768383/users/49cf3c2240bf4d3482b6c031ada51ffe/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:30.742273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac6bfe-882a-4f1c-bc31-72230ba15784 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:30.748680 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac6bfe-882a-4f1c-bc31-72230ba15784 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:30.752620 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac6bfe-882a-4f1c-bc31-72230ba15784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:30.808861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80079f5c-d5f4-44f7-bf53-45cc72defe9a None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:30.869545 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-76ac6bfe-882a-4f1c-bc31-72230ba15784 None None] MFA Rules not processed for user `49cf3c2240bf4d3482b6c031ada51ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:30.902897 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-76ac6bfe-882a-4f1c-bc31-72230ba15784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:30.979238 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6ed31-554c-44ed-a72c-8d032705d933 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:30.980970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6ed31-554c-44ed-a72c-8d032705d933 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:30.981600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6ed31-554c-44ed-a72c-8d032705d933 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:30.983154 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6ed31-554c-44ed-a72c-8d032705d933 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:31.333924 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:31.340227 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 975/1943] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 601 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:31.341252 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf6ed31-554c-44ed-a72c-8d032705d933 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:31.344211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf6ed31-554c-44ed-a72c-8d032705d933 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:31.345568 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac6bfe-882a-4f1c-bc31-72230ba15784 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:31.377948 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac6bfe-882a-4f1c-bc31-72230ba15784 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:31.394714 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c5bb0-8f36-4cb4-976e-7ce684466d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:31.395366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c5bb0-8f36-4cb4-976e-7ce684466d6b admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:31.399499 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c5bb0-8f36-4cb4-976e-7ce684466d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:31.400182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c5bb0-8f36-4cb4-976e-7ce684466d6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:31.403269 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c5bb0-8f36-4cb4-976e-7ce684466d6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:31.407944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c5bb0-8f36-4cb4-976e-7ce684466d6b admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:31.410715 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 969/1944] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 888 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:13:31.418112 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:31.484851 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 976/1945] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:13:31 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:31.492466 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6ed31-554c-44ed-a72c-8d032705d933 service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:31.525539 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6ed31-554c-44ed-a72c-8d032705d933 service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:31.730808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4b7b7-55eb-4a58-9d71-221e99460dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:31.732735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4b7b7-55eb-4a58-9d71-221e99460dbb admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:31.733393 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4b7b7-55eb-4a58-9d71-221e99460dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:31.734153 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4b7b7-55eb-4a58-9d71-221e99460dbb admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:31.736250 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4b7b7-55eb-4a58-9d71-221e99460dbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:31.740348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4b7b7-55eb-4a58-9d71-221e99460dbb admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:31.806475 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 970/1946] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:13:31 2022] POST /identity/v3/users => generated 299 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:31.811214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c5bb0-8f36-4cb4-976e-7ce684466d6b admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:31.824176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c5bb0-8f36-4cb4-976e-7ce684466d6b admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:31.832793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e587a256-b597-44db-8be3-7ffe77d91aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:31.833952 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587a256-b597-44db-8be3-7ffe77d91aea admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:31.834481 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587a256-b597-44db-8be3-7ffe77d91aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:31.834945 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587a256-b597-44db-8be3-7ffe77d91aea admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:31.836156 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e587a256-b597-44db-8be3-7ffe77d91aea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:31.839425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e587a256-b597-44db-8be3-7ffe77d91aea admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:31.866393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 977/1947] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:13:31.869120 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4b7b7-55eb-4a58-9d71-221e99460dbb admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:31.891017 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4b7b7-55eb-4a58-9d71-221e99460dbb admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:31.899732 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb515b-c7b3-41f4-b71a-217001be16d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:31.902656 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb515b-c7b3-41f4-b71a-217001be16d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:31.902940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb515b-c7b3-41f4-b71a-217001be16d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:31.903313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb515b-c7b3-41f4-b71a-217001be16d7 admin admin] PATH_INFO: `/v3/projects/f4b58b0aaee74f9da75bb8a98e8234b6/users/44425b91e6be4475ab3f16da81985ce6/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:31.932765 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fb515b-c7b3-41f4-b71a-217001be16d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b58b0aaee74f9da75bb8a98e8234b6, user_id=44425b91e6be4475ab3f16da81985ce6, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:31.939193 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fb515b-c7b3-41f4-b71a-217001be16d7 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:31.959733 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 971/1948] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:31 2022] PUT /identity/v3/projects/f4b58b0aaee74f9da75bb8a98e8234b6/users/44425b91e6be4475ab3f16da81985ce6/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:13:31.964810 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e587a256-b597-44db-8be3-7ffe77d91aea admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:31.976617 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e587a256-b597-44db-8be3-7ffe77d91aea admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:32.077985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb515b-c7b3-41f4-b71a-217001be16d7 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:32.081240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-66fb515b-c7b3-41f4-b71a-217001be16d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:32.121116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519db541-e953-4c0a-93c8-f79b57aa3e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:32.121428 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519db541-e953-4c0a-93c8-f79b57aa3e2d admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:32.121915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519db541-e953-4c0a-93c8-f79b57aa3e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:32.122373 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519db541-e953-4c0a-93c8-f79b57aa3e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:32.123596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519db541-e953-4c0a-93c8-f79b57aa3e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:32.129383 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519db541-e953-4c0a-93c8-f79b57aa3e2d admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:32.168283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 978/1949] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:32.180311 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519db541-e953-4c0a-93c8-f79b57aa3e2d admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:32.207468 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519db541-e953-4c0a-93c8-f79b57aa3e2d admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:32.215608 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31650242-6b3d-46f6-92ee-2ebdd511aaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:32.217984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31650242-6b3d-46f6-92ee-2ebdd511aaf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:32.217984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31650242-6b3d-46f6-92ee-2ebdd511aaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:32.217984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31650242-6b3d-46f6-92ee-2ebdd511aaf6 admin admin] PATH_INFO: `/v3/projects/f4b58b0aaee74f9da75bb8a98e8234b6/users/44425b91e6be4475ab3f16da81985ce6/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:32.219842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31650242-6b3d-46f6-92ee-2ebdd511aaf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b58b0aaee74f9da75bb8a98e8234b6, user_id=44425b91e6be4475ab3f16da81985ce6, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:32.222599 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31650242-6b3d-46f6-92ee-2ebdd511aaf6 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:32.234937 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb515b-c7b3-41f4-b71a-217001be16d7 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:32.270021 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 979/1950] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:32 2022] PUT /identity/v3/projects/f4b58b0aaee74f9da75bb8a98e8234b6/users/44425b91e6be4475ab3f16da81985ce6/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:32.289546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e00554-3b56-4fd9-bd5d-0f165ed4c03e None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:32.290245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e00554-3b56-4fd9-bd5d-0f165ed4c03e None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:32.290842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e00554-3b56-4fd9-bd5d-0f165ed4c03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:32.375311 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-f1e00554-3b56-4fd9-bd5d-0f165ed4c03e None None] MFA Rules not processed for user `44425b91e6be4475ab3f16da81985ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:32.378213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f1e00554-3b56-4fd9-bd5d-0f165ed4c03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:32.415078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab5f19-290e-429a-a6ba-a0d115a5be34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:32.416159 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab5f19-290e-429a-a6ba-a0d115a5be34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:32.417279 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab5f19-290e-429a-a6ba-a0d115a5be34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:32.417797 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab5f19-290e-429a-a6ba-a0d115a5be34 admin admin] PATH_INFO: `/v3/projects/b1aae31d82a94679acd5afb18799eb9d` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:32.425086 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ab5f19-290e-429a-a6ba-a0d115a5be34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1aae31d82a94679acd5afb18799eb9d)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:32.425086 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ab5f19-290e-429a-a6ba-a0d115a5be34 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:32.451930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:32.563690 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d0ab5f19-290e-429a-a6ba-a0d115a5be34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1aae31d82a94679acd5afb18799eb9d', 'request_id': 'req-d0ab5f19-290e-429a-a6ba-a0d115a5be34'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:32.589605 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d0ab5f19-290e-429a-a6ba-a0d115a5be34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1aae31d82a94679acd5afb18799eb9d', 'request_id': 'req-d0ab5f19-290e-429a-a6ba-a0d115a5be34'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:32.621972 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d0ab5f19-290e-429a-a6ba-a0d115a5be34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1aae31d82a94679acd5afb18799eb9d', 'request_id': 'req-d0ab5f19-290e-429a-a6ba-a0d115a5be34'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:32.633308 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 980/1951] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:32 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:13:32.659497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d0ab5f19-290e-429a-a6ba-a0d115a5be34 admin admin] The token cache is being invalidate because project b1aae31d82a94679acd5afb18799eb9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:13:32.659984 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d0ab5f19-290e-429a-a6ba-a0d115a5be34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:32.661239 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d0ab5f19-290e-429a-a6ba-a0d115a5be34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:32.672443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e00554-3b56-4fd9-bd5d-0f165ed4c03e None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:32.673203 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 972/1952] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:13:32 2022] DELETE /identity/v3/projects/b1aae31d82a94679acd5afb18799eb9d => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:13:32.675071 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f1e00554-3b56-4fd9-bd5d-0f165ed4c03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:32.679786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab5f19-290e-429a-a6ba-a0d115a5be34 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:32.691387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d0ab5f19-290e-429a-a6ba-a0d115a5be34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:32.875083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab5f19-290e-429a-a6ba-a0d115a5be34 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:32.884118 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da9b4b-b599-4c21-b739-42a15f045373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:32.885289 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da9b4b-b599-4c21-b739-42a15f045373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:32.885805 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da9b4b-b599-4c21-b739-42a15f045373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:32.886364 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da9b4b-b599-4c21-b739-42a15f045373 admin admin] PATH_INFO: `/v3/users/28cf2abf2d3f42b9afea0e121ed1ad25` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:32.889030 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da9b4b-b599-4c21-b739-42a15f045373 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28cf2abf2d3f42b9afea0e121ed1ad25)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:32.890472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:32.894599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da9b4b-b599-4c21-b739-42a15f045373 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:32.924151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e00554-3b56-4fd9-bd5d-0f165ed4c03e None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:33.016988 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c8da9b4b-b599-4c21-b739-42a15f045373 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28cf2abf2d3f42b9afea0e121ed1ad25'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:33.038750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c8da9b4b-b599-4c21-b739-42a15f045373 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28cf2abf2d3f42b9afea0e121ed1ad25'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:33.073593 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c8da9b4b-b599-4c21-b739-42a15f045373 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28cf2abf2d3f42b9afea0e121ed1ad25'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:33.078040 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640aa9b-1406-410a-8bab-8e5d36c874e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:33.079035 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640aa9b-1406-410a-8bab-8e5d36c874e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:33.079662 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640aa9b-1406-410a-8bab-8e5d36c874e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:33.080112 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640aa9b-1406-410a-8bab-8e5d36c874e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:33.083494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c640aa9b-1406-410a-8bab-8e5d36c874e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:33.100519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c8da9b4b-b599-4c21-b739-42a15f045373 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28cf2abf2d3f42b9afea0e121ed1ad25'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:33.102357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c8da9b4b-b599-4c21-b739-42a15f045373 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28cf2abf2d3f42b9afea0e121ed1ad25'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:33.130128 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 973/1953] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:13:32 2022] DELETE /identity/v3/users/28cf2abf2d3f42b9afea0e121ed1ad25 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:33.225571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640aa9b-1406-410a-8bab-8e5d36c874e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:33.228426 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640aa9b-1406-410a-8bab-8e5d36c874e1 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:33.266802 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 981/1954] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:13:33.812903 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da9b4b-b599-4c21-b739-42a15f045373 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:33.815196 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c8da9b4b-b599-4c21-b739-42a15f045373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:33.840351 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:33.947423 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da9b4b-b599-4c21-b739-42a15f045373 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:33.954366 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f83ed-827f-44c4-9b3a-f68fea00f000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:33.955173 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f83ed-827f-44c4-9b3a-f68fea00f000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:33.955547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f83ed-827f-44c4-9b3a-f68fea00f000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:33.955880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f83ed-827f-44c4-9b3a-f68fea00f000 admin admin] PATH_INFO: `/v3/projects/c3c7ba4578c54018962163856667f3f4` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:33.957376 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f83ed-827f-44c4-9b3a-f68fea00f000 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3c7ba4578c54018962163856667f3f4)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:33.959467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f83ed-827f-44c4-9b3a-f68fea00f000 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:34.041686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-958f83ed-827f-44c4-9b3a-f68fea00f000 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3c7ba4578c54018962163856667f3f4', 'request_id': 'req-958f83ed-827f-44c4-9b3a-f68fea00f000'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:34.059703 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-958f83ed-827f-44c4-9b3a-f68fea00f000 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3c7ba4578c54018962163856667f3f4', 'request_id': 'req-958f83ed-827f-44c4-9b3a-f68fea00f000'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:34.106186 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-958f83ed-827f-44c4-9b3a-f68fea00f000 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3c7ba4578c54018962163856667f3f4', 'request_id': 'req-958f83ed-827f-44c4-9b3a-f68fea00f000'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:34.141244 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-958f83ed-827f-44c4-9b3a-f68fea00f000 admin admin] The token cache is being invalidate because project c3c7ba4578c54018962163856667f3f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:13:34.141244 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-958f83ed-827f-44c4-9b3a-f68fea00f000 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:34.142066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-958f83ed-827f-44c4-9b3a-f68fea00f000 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:34.145855 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 974/1955] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:13:33 2022] DELETE /identity/v3/projects/c3c7ba4578c54018962163856667f3f4 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:34.151106 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482e50d-e056-4f50-aa16-e7114c716d0d None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:34.152046 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482e50d-e056-4f50-aa16-e7114c716d0d None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:34.152552 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482e50d-e056-4f50-aa16-e7114c716d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:34.192084 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-c482e50d-e056-4f50-aa16-e7114c716d0d None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:34.193339 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c482e50d-e056-4f50-aa16-e7114c716d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:34.347090 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 982/1956] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:34.358212 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffa427-7d09-45a0-913f-19820a73cefc None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:34.359442 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffa427-7d09-45a0-913f-19820a73cefc None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:34.359442 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffa427-7d09-45a0-913f-19820a73cefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:34.396354 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-f9ffa427-7d09-45a0-913f-19820a73cefc None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:34.398492 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-f9ffa427-7d09-45a0-913f-19820a73cefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:34.413432 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 975/1957] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:13:34.419101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482e50d-e056-4f50-aa16-e7114c716d0d None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:34.431760 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482e50d-e056-4f50-aa16-e7114c716d0d None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:34.439407 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a364e-329e-4640-9c86-86741e6c0e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:34.440066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a364e-329e-4640-9c86-86741e6c0e72 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:34.442611 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a364e-329e-4640-9c86-86741e6c0e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:34.443234 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a364e-329e-4640-9c86-86741e6c0e72 admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:34.444333 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a364e-329e-4640-9c86-86741e6c0e72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:13:34.444939 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a364e-329e-4640-9c86-86741e6c0e72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:34.447305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a364e-329e-4640-9c86-86741e6c0e72 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:34.479733 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 983/1958] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:13:34.488562 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffa427-7d09-45a0-913f-19820a73cefc None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:34.506844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffa427-7d09-45a0-913f-19820a73cefc None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:34.533537 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c28014-69f3-49d1-abcf-3a312acb16de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:34.534320 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c28014-69f3-49d1-abcf-3a312acb16de admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:34.534611 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c28014-69f3-49d1-abcf-3a312acb16de admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:34.535046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c28014-69f3-49d1-abcf-3a312acb16de admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:34.535840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c28014-69f3-49d1-abcf-3a312acb16de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:34.539334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c28014-69f3-49d1-abcf-3a312acb16de admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:34.636765 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 976/1959] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:13:34 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:34.643874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a364e-329e-4640-9c86-86741e6c0e72 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:34.663317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a364e-329e-4640-9c86-86741e6c0e72 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:34.746929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:34.795712 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea81eb0-32ad-48e5-9c65-5468163c2d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:34.796344 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea81eb0-32ad-48e5-9c65-5468163c2d74 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:34.796628 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea81eb0-32ad-48e5-9c65-5468163c2d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:34.796901 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea81eb0-32ad-48e5-9c65-5468163c2d74 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:34.797618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea81eb0-32ad-48e5-9c65-5468163c2d74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:34.800563 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea81eb0-32ad-48e5-9c65-5468163c2d74 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:34.839361 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 984/1960] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:13:34 2022] POST /identity/v3/users => generated 290 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:13:34.844289 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c28014-69f3-49d1-abcf-3a312acb16de admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:34.859906 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c28014-69f3-49d1-abcf-3a312acb16de admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:34.865251 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3b914-2c44-4824-bd79-afa0aff733ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:34.865891 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3b914-2c44-4824-bd79-afa0aff733ed admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:34.866170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3b914-2c44-4824-bd79-afa0aff733ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:34.866437 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3b914-2c44-4824-bd79-afa0aff733ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:34.867240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3b914-2c44-4824-bd79-afa0aff733ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:34.868998 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3b914-2c44-4824-bd79-afa0aff733ed admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:34.880992 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 977/1961] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:34.886466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea81eb0-32ad-48e5-9c65-5468163c2d74 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:34.901996 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea81eb0-32ad-48e5-9c65-5468163c2d74 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:34.906921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c8474-591e-47a3-82e6-72e990998920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:34.907543 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c8474-591e-47a3-82e6-72e990998920 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:34.907832 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c8474-591e-47a3-82e6-72e990998920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:34.908107 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c8474-591e-47a3-82e6-72e990998920 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:34.908741 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c8474-591e-47a3-82e6-72e990998920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:34.911099 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c8474-591e-47a3-82e6-72e990998920 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:34.925980 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 985/1962] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:34.928962 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3b914-2c44-4824-bd79-afa0aff733ed admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:34.929183 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c8474-591e-47a3-82e6-72e990998920 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:34.929315 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-90c3b914-2c44-4824-bd79-afa0aff733ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:34.938710 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c8474-591e-47a3-82e6-72e990998920 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:34.943425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3b914-2c44-4824-bd79-afa0aff733ed admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:34.943759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed36a3-063e-4522-81e8-a4e5df8bb3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:34.944436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed36a3-063e-4522-81e8-a4e5df8bb3c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:34.944833 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed36a3-063e-4522-81e8-a4e5df8bb3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:34.944999 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed36a3-063e-4522-81e8-a4e5df8bb3c1 admin admin] PATH_INFO: `/v3/projects/dc52b5d31e324b1ab41a3ee62d86c112/users/00ace6f031034f44912ab7fd6e075f56/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:34.950093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724e0d3-77a5-43a5-b9eb-c168059b1fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:34.950093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724e0d3-77a5-43a5-b9eb-c168059b1fff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:34.950093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724e0d3-77a5-43a5-b9eb-c168059b1fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:34.950646 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724e0d3-77a5-43a5-b9eb-c168059b1fff admin admin] PATH_INFO: `/v3/users/49cf3c2240bf4d3482b6c031ada51ffe` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:34.952824 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724e0d3-77a5-43a5-b9eb-c168059b1fff admin admin] RBAC: Authorizing `identity:delete_user(user_id=49cf3c2240bf4d3482b6c031ada51ffe)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:34.954901 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724e0d3-77a5-43a5-b9eb-c168059b1fff admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:34.966863 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed36a3-063e-4522-81e8-a4e5df8bb3c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc52b5d31e324b1ab41a3ee62d86c112, user_id=00ace6f031034f44912ab7fd6e075f56, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:34.969780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed36a3-063e-4522-81e8-a4e5df8bb3c1 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:34.991443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 986/1963] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:34 2022] PUT /identity/v3/projects/dc52b5d31e324b1ab41a3ee62d86c112/users/00ace6f031034f44912ab7fd6e075f56/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:35.004640 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b27bb-eccd-4540-99e3-62d91ee5e75f None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:35.004980 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b27bb-eccd-4540-99e3-62d91ee5e75f None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:35.005305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b27bb-eccd-4540-99e3-62d91ee5e75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:35.023025 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d724e0d3-77a5-43a5-b9eb-c168059b1fff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49cf3c2240bf4d3482b6c031ada51ffe'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:35.035671 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d724e0d3-77a5-43a5-b9eb-c168059b1fff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49cf3c2240bf4d3482b6c031ada51ffe'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:35.039862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-f04b27bb-eccd-4540-99e3-62d91ee5e75f None None] MFA Rules not processed for user `00ace6f031034f44912ab7fd6e075f56`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:35.041223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f04b27bb-eccd-4540-99e3-62d91ee5e75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:35.046533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:35.061986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d724e0d3-77a5-43a5-b9eb-c168059b1fff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49cf3c2240bf4d3482b6c031ada51ffe'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:35.073174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d724e0d3-77a5-43a5-b9eb-c168059b1fff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49cf3c2240bf4d3482b6c031ada51ffe'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:35.074068 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d724e0d3-77a5-43a5-b9eb-c168059b1fff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49cf3c2240bf4d3482b6c031ada51ffe'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:35.085237 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 978/1964] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:13:34 2022] DELETE /identity/v3/users/49cf3c2240bf4d3482b6c031ada51ffe => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:35.086822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c578597-ca39-448b-91ff-1dd11dc953f3 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:35.088309 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c578597-ca39-448b-91ff-1dd11dc953f3 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:35.088309 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c578597-ca39-448b-91ff-1dd11dc953f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:35.117564 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-5c578597-ca39-448b-91ff-1dd11dc953f3 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:35.118899 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-5c578597-ca39-448b-91ff-1dd11dc953f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:35.142221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 987/1965] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:13:35.147040 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b27bb-eccd-4540-99e3-62d91ee5e75f None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:35.148728 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f04b27bb-eccd-4540-99e3-62d91ee5e75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:35.218675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 979/1966] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:35.233397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c578597-ca39-448b-91ff-1dd11dc953f3 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:35.234040 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b27bb-eccd-4540-99e3-62d91ee5e75f None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:35.236294 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-5c578597-ca39-448b-91ff-1dd11dc953f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:35.239164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1431252-00b5-42b9-9ede-f659b830e5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:35.239779 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1431252-00b5-42b9-9ede-f659b830e5c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:35.240180 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1431252-00b5-42b9-9ede-f659b830e5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:35.240335 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1431252-00b5-42b9-9ede-f659b830e5c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:35.240970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1431252-00b5-42b9-9ede-f659b830e5c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:35.243190 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1431252-00b5-42b9-9ede-f659b830e5c2 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:35.302226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 988/1967] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:13:35 2022] POST /identity/v3/projects => generated 381 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:35.304526 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1431252-00b5-42b9-9ede-f659b830e5c2 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:35.317467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:35.323543 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c578597-ca39-448b-91ff-1dd11dc953f3 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:35.387311 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d7a00-a9bb-45cd-9636-49351173d585 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:35.387929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d7a00-a9bb-45cd-9636-49351173d585 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:35.388244 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d7a00-a9bb-45cd-9636-49351173d585 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:35.388515 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d7a00-a9bb-45cd-9636-49351173d585 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:35.416488 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1431252-00b5-42b9-9ede-f659b830e5c2 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:35.423148 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc29bca-ea0f-446f-9bc2-5d7308d8c8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:35.424095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc29bca-ea0f-446f-9bc2-5d7308d8c8df admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:35.424095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc29bca-ea0f-446f-9bc2-5d7308d8c8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:35.424357 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc29bca-ea0f-446f-9bc2-5d7308d8c8df admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:35.425079 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc29bca-ea0f-446f-9bc2-5d7308d8c8df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:35.427393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc29bca-ea0f-446f-9bc2-5d7308d8c8df admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:35.467356 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 989/1968] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:13:35 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:35.469629 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:35.473935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc29bca-ea0f-446f-9bc2-5d7308d8c8df admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:35.486754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d7a00-a9bb-45cd-9636-49351173d585 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:35.487067 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc29bca-ea0f-446f-9bc2-5d7308d8c8df admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:35.488419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d7a00-a9bb-45cd-9636-49351173d585 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:35.493356 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0789a-1457-41fc-ac2f-535f17cdaebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:35.494010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0789a-1457-41fc-ac2f-535f17cdaebe admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:35.494293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0789a-1457-41fc-ac2f-535f17cdaebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:35.494591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0789a-1457-41fc-ac2f-535f17cdaebe admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:35.495256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0789a-1457-41fc-ac2f-535f17cdaebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:35.496879 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0789a-1457-41fc-ac2f-535f17cdaebe admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:35.504379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 980/1969] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:13:35.511683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 990/1970] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:35.513897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d7a00-a9bb-45cd-9636-49351173d585 service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:35.530562 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d7a00-a9bb-45cd-9636-49351173d585 service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:35.538195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6cc14-912d-4471-b432-9b8988532e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:35.539283 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6cc14-912d-4471-b432-9b8988532e1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:35.539749 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6cc14-912d-4471-b432-9b8988532e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:35.540218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6cc14-912d-4471-b432-9b8988532e1d admin admin] PATH_INFO: `/v3/projects/d948be285b1844438c2c583d62b95f8f/users/2175699ac5564d8c845cdfb1cc2f065f/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:35.570195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6cc14-912d-4471-b432-9b8988532e1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d948be285b1844438c2c583d62b95f8f, user_id=2175699ac5564d8c845cdfb1cc2f065f, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:35.575842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6cc14-912d-4471-b432-9b8988532e1d admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:35.596390 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 981/1971] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:35 2022] PUT /identity/v3/projects/d948be285b1844438c2c583d62b95f8f/users/2175699ac5564d8c845cdfb1cc2f065f/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:35.600563 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0789a-1457-41fc-ac2f-535f17cdaebe admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:35.612307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0789a-1457-41fc-ac2f-535f17cdaebe admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:35.682346 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4019a8d-f0f4-4b78-947b-e65c22729a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:35.682895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4019a8d-f0f4-4b78-947b-e65c22729a94 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:35.683185 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4019a8d-f0f4-4b78-947b-e65c22729a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:35.683452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4019a8d-f0f4-4b78-947b-e65c22729a94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:35.684121 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4019a8d-f0f4-4b78-947b-e65c22729a94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:35.685999 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4019a8d-f0f4-4b78-947b-e65c22729a94 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:35.701523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6cc14-912d-4471-b432-9b8988532e1d admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:35.717911 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 991/1972] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:35.727852 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6cc14-912d-4471-b432-9b8988532e1d admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:35.737197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e74bf-7895-41bd-af51-cab5524288a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:35.738288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e74bf-7895-41bd-af51-cab5524288a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:35.738774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e74bf-7895-41bd-af51-cab5524288a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:35.739286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e74bf-7895-41bd-af51-cab5524288a9 admin admin] PATH_INFO: `/v3/projects/d948be285b1844438c2c583d62b95f8f/users/2175699ac5564d8c845cdfb1cc2f065f/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:35.742976 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e74bf-7895-41bd-af51-cab5524288a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d948be285b1844438c2c583d62b95f8f, user_id=2175699ac5564d8c845cdfb1cc2f065f, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:35.748305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e74bf-7895-41bd-af51-cab5524288a9 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:35.762902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 982/1973] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:35 2022] PUT /identity/v3/projects/d948be285b1844438c2c583d62b95f8f/users/2175699ac5564d8c845cdfb1cc2f065f/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:13:35.773699 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9b434-2305-4312-b118-4b36139986e0 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:35.774009 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9b434-2305-4312-b118-4b36139986e0 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:35.774286 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9b434-2305-4312-b118-4b36139986e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:35.807960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-25b9b434-2305-4312-b118-4b36139986e0 None None] MFA Rules not processed for user `2175699ac5564d8c845cdfb1cc2f065f`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:35.809269 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-25b9b434-2305-4312-b118-4b36139986e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:35.924427 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 992/1974] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:35.953978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e74bf-7895-41bd-af51-cab5524288a9 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:35.956555 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9b6e74bf-7895-41bd-af51-cab5524288a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:36.017092 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9b434-2305-4312-b118-4b36139986e0 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:36.019724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-25b9b434-2305-4312-b118-4b36139986e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:36.040169 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:36.070889 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e74bf-7895-41bd-af51-cab5524288a9 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:36.076794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0d494-9997-4c09-8278-aff976e4db55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:36.077608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0d494-9997-4c09-8278-aff976e4db55 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:36.078507 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0d494-9997-4c09-8278-aff976e4db55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:36.079026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0d494-9997-4c09-8278-aff976e4db55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:36.109653 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0d494-9997-4c09-8278-aff976e4db55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:36.112640 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0d494-9997-4c09-8278-aff976e4db55 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:36.128553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9b434-2305-4312-b118-4b36139986e0 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:36.131345 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 983/1975] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:13:36.135380 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d55ef-84fa-4318-90d7-e676ad354a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:36.136506 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d55ef-84fa-4318-90d7-e676ad354a7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:36.137079 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d55ef-84fa-4318-90d7-e676ad354a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:36.137634 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d55ef-84fa-4318-90d7-e676ad354a7f admin admin] PATH_INFO: `/v3/projects/92c2b524a96e42df9a48390eea768383` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:36.139905 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d55ef-84fa-4318-90d7-e676ad354a7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=92c2b524a96e42df9a48390eea768383)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:36.143582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d55ef-84fa-4318-90d7-e676ad354a7f admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:36.230808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-ef5d55ef-84fa-4318-90d7-e676ad354a7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92c2b524a96e42df9a48390eea768383', 'request_id': 'req-ef5d55ef-84fa-4318-90d7-e676ad354a7f'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:36.244627 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-ef5d55ef-84fa-4318-90d7-e676ad354a7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92c2b524a96e42df9a48390eea768383', 'request_id': 'req-ef5d55ef-84fa-4318-90d7-e676ad354a7f'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:36.264595 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-ef5d55ef-84fa-4318-90d7-e676ad354a7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92c2b524a96e42df9a48390eea768383', 'request_id': 'req-ef5d55ef-84fa-4318-90d7-e676ad354a7f'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:36.299866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-ef5d55ef-84fa-4318-90d7-e676ad354a7f admin admin] The token cache is being invalidate because project 92c2b524a96e42df9a48390eea768383 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:13:36.300380 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-ef5d55ef-84fa-4318-90d7-e676ad354a7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:36.301448 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-ef5d55ef-84fa-4318-90d7-e676ad354a7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:36.306205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 993/1976] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:13:36 2022] DELETE /identity/v3/projects/92c2b524a96e42df9a48390eea768383 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:36.311487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0d494-9997-4c09-8278-aff976e4db55 service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:36.327571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0d494-9997-4c09-8278-aff976e4db55 service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:36.463855 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c36ca-6c31-406c-ae25-e59f25be245b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:36.465327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c36ca-6c31-406c-ae25-e59f25be245b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:36.465942 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c36ca-6c31-406c-ae25-e59f25be245b admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:36.466624 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c36ca-6c31-406c-ae25-e59f25be245b admin admin] PATH_INFO: `/v3/users/44425b91e6be4475ab3f16da81985ce6` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:36.468898 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c36ca-6c31-406c-ae25-e59f25be245b admin admin] RBAC: Authorizing `identity:delete_user(user_id=44425b91e6be4475ab3f16da81985ce6)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:36.473171 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c36ca-6c31-406c-ae25-e59f25be245b admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:36.536860 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f28c36ca-6c31-406c-ae25-e59f25be245b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44425b91e6be4475ab3f16da81985ce6'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:36.541276 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:36.547144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f28c36ca-6c31-406c-ae25-e59f25be245b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44425b91e6be4475ab3f16da81985ce6'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:36.566283 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f28c36ca-6c31-406c-ae25-e59f25be245b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44425b91e6be4475ab3f16da81985ce6'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:36.577261 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f28c36ca-6c31-406c-ae25-e59f25be245b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44425b91e6be4475ab3f16da81985ce6'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:36.578246 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f28c36ca-6c31-406c-ae25-e59f25be245b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44425b91e6be4475ab3f16da81985ce6'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:36.585314 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 984/1977] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:13:36 2022] DELETE /identity/v3/users/44425b91e6be4475ab3f16da81985ce6 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:13:36.714400 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d55ef-84fa-4318-90d7-e676ad354a7f admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:36.718497 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ef5d55ef-84fa-4318-90d7-e676ad354a7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:36.742402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d55ef-84fa-4318-90d7-e676ad354a7f admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:36.886896 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664933cf-66ca-4f39-84c7-57912a5ad07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:36.888026 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664933cf-66ca-4f39-84c7-57912a5ad07d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:36.889284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664933cf-66ca-4f39-84c7-57912a5ad07d admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:36.889624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664933cf-66ca-4f39-84c7-57912a5ad07d admin admin] PATH_INFO: `/v3/users/00ace6f031034f44912ab7fd6e075f56` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:36.891881 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664933cf-66ca-4f39-84c7-57912a5ad07d admin admin] RBAC: Authorizing `identity:delete_user(user_id=00ace6f031034f44912ab7fd6e075f56)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:36.896093 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664933cf-66ca-4f39-84c7-57912a5ad07d admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:36.970751 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c36ca-6c31-406c-ae25-e59f25be245b admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:36.972306 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-f28c36ca-6c31-406c-ae25-e59f25be245b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:36.972421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:36.987210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-664933cf-66ca-4f39-84c7-57912a5ad07d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00ace6f031034f44912ab7fd6e075f56'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:36.995000 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c36ca-6c31-406c-ae25-e59f25be245b admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:37.006691 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8fd374-2b27-472e-a854-e867b52064e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:37.007351 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fd374-2b27-472e-a854-e867b52064e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:37.007636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fd374-2b27-472e-a854-e867b52064e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:37.007914 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fd374-2b27-472e-a854-e867b52064e6 admin admin] PATH_INFO: `/v3/projects/f4b58b0aaee74f9da75bb8a98e8234b6` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:37.009589 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8fd374-2b27-472e-a854-e867b52064e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4b58b0aaee74f9da75bb8a98e8234b6)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:37.011916 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8fd374-2b27-472e-a854-e867b52064e6 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:37.060605 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-664933cf-66ca-4f39-84c7-57912a5ad07d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00ace6f031034f44912ab7fd6e075f56'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:37.070500 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-664933cf-66ca-4f39-84c7-57912a5ad07d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00ace6f031034f44912ab7fd6e075f56'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:37.079625 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-664933cf-66ca-4f39-84c7-57912a5ad07d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00ace6f031034f44912ab7fd6e075f56'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:37.080369 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-664933cf-66ca-4f39-84c7-57912a5ad07d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00ace6f031034f44912ab7fd6e075f56'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:37.087845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 994/1978] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:13:36 2022] DELETE /identity/v3/users/00ace6f031034f44912ab7fd6e075f56 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:37.092509 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb28e05-e6ac-478a-8301-35010acbc80e None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:37.093243 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb28e05-e6ac-478a-8301-35010acbc80e None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:37.095072 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb28e05-e6ac-478a-8301-35010acbc80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:37.115469 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-ca8fd374-2b27-472e-a854-e867b52064e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4b58b0aaee74f9da75bb8a98e8234b6', 'request_id': 'req-ca8fd374-2b27-472e-a854-e867b52064e6'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:37.119942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-2eb28e05-e6ac-478a-8301-35010acbc80e None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:37.121271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-2eb28e05-e6ac-478a-8301-35010acbc80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:37.124910 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-ca8fd374-2b27-472e-a854-e867b52064e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4b58b0aaee74f9da75bb8a98e8234b6', 'request_id': 'req-ca8fd374-2b27-472e-a854-e867b52064e6'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:37.134469 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-ca8fd374-2b27-472e-a854-e867b52064e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4b58b0aaee74f9da75bb8a98e8234b6', 'request_id': 'req-ca8fd374-2b27-472e-a854-e867b52064e6'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:37.166130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-ca8fd374-2b27-472e-a854-e867b52064e6 admin admin] The token cache is being invalidate because project f4b58b0aaee74f9da75bb8a98e8234b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:13:37.166681 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-ca8fd374-2b27-472e-a854-e867b52064e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:37.167636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-ca8fd374-2b27-472e-a854-e867b52064e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:37.170136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 985/1979] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:13:36 2022] DELETE /identity/v3/projects/f4b58b0aaee74f9da75bb8a98e8234b6 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:37.176298 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93617e2-0e4f-4de7-9f17-d1692a4f0afb None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:37.177145 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93617e2-0e4f-4de7-9f17-d1692a4f0afb None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:37.177662 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93617e2-0e4f-4de7-9f17-d1692a4f0afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:37.212138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 995/1980] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:37.212309 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-e93617e2-0e4f-4de7-9f17-d1692a4f0afb None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:37.212309 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-e93617e2-0e4f-4de7-9f17-d1692a4f0afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:37.220157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb28e05-e6ac-478a-8301-35010acbc80e None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:37.221557 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-2eb28e05-e6ac-478a-8301-35010acbc80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:37.227058 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 986/1981] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:37.238309 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18baaf-7502-4090-a4ed-624db430032c None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:37.238672 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18baaf-7502-4090-a4ed-624db430032c None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:37.239202 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18baaf-7502-4090-a4ed-624db430032c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:37.271430 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-2b18baaf-7502-4090-a4ed-624db430032c None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:37.273276 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-2b18baaf-7502-4090-a4ed-624db430032c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:37.287647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 987/1982] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:13:37.295316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18baaf-7502-4090-a4ed-624db430032c None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:37.311840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18baaf-7502-4090-a4ed-624db430032c None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:37.317439 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee9016-c248-4264-a688-67a397348486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:37.318097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee9016-c248-4264-a688-67a397348486 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:37.318952 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee9016-c248-4264-a688-67a397348486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:37.319685 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee9016-c248-4264-a688-67a397348486 admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:37.320510 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee9016-c248-4264-a688-67a397348486 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:13:37.320510 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee9016-c248-4264-a688-67a397348486 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:37.322183 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee9016-c248-4264-a688-67a397348486 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:37.324217 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb28e05-e6ac-478a-8301-35010acbc80e None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:37.328862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed472c1c-042b-440c-ae22-28f65c3ef824 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:37.329478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed472c1c-042b-440c-ae22-28f65c3ef824 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:37.329774 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed472c1c-042b-440c-ae22-28f65c3ef824 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:37.330198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed472c1c-042b-440c-ae22-28f65c3ef824 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:37.336336 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 988/1983] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:13:37.340665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee9016-c248-4264-a688-67a397348486 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:37.342713 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed472c1c-042b-440c-ae22-28f65c3ef824 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:37.344363 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed472c1c-042b-440c-ae22-28f65c3ef824 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:37.351396 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee9016-c248-4264-a688-67a397348486 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:37.356179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 996/1984] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:13:37.356462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0a79b-c95e-48be-8ca6-7f3b43ebf0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:37.357601 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0a79b-c95e-48be-8ca6-7f3b43ebf0e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:37.358023 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0a79b-c95e-48be-8ca6-7f3b43ebf0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:37.358592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0a79b-c95e-48be-8ca6-7f3b43ebf0e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:37.359275 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f0a79b-c95e-48be-8ca6-7f3b43ebf0e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:37.361199 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f0a79b-c95e-48be-8ca6-7f3b43ebf0e8 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:37.392828 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 989/1985] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:13:37 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:13:37.397179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed472c1c-042b-440c-ae22-28f65c3ef824 service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:37.409252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed472c1c-042b-440c-ae22-28f65c3ef824 service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:37.501543 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb6d12-9a50-4e01-969e-c541da89bbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:37.502350 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb6d12-9a50-4e01-969e-c541da89bbc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:37.502639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb6d12-9a50-4e01-969e-c541da89bbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:37.502907 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb6d12-9a50-4e01-969e-c541da89bbc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:37.503566 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb6d12-9a50-4e01-969e-c541da89bbc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:37.505600 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb6d12-9a50-4e01-969e-c541da89bbc8 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:37.520817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:37.545766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 997/1986] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:13:37 2022] POST /identity/v3/users => generated 286 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:37.547420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0a79b-c95e-48be-8ca6-7f3b43ebf0e8 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:37.568426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0a79b-c95e-48be-8ca6-7f3b43ebf0e8 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:37.576718 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3388b-9693-4b14-84e3-1b99372690f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:37.577421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3388b-9693-4b14-84e3-1b99372690f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:37.577773 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3388b-9693-4b14-84e3-1b99372690f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:37.578023 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3388b-9693-4b14-84e3-1b99372690f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:37.578652 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c3388b-9693-4b14-84e3-1b99372690f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:37.580617 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c3388b-9693-4b14-84e3-1b99372690f1 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:37.595728 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 990/1987] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:37.601013 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb6d12-9a50-4e01-969e-c541da89bbc8 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:37.616610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb6d12-9a50-4e01-969e-c541da89bbc8 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:37.621575 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c9035-180f-4327-b558-01340cc903d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:37.622225 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c9035-180f-4327-b558-01340cc903d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:37.622513 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c9035-180f-4327-b558-01340cc903d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:37.623398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c9035-180f-4327-b558-01340cc903d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:37.624098 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c9035-180f-4327-b558-01340cc903d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:37.625874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c9035-180f-4327-b558-01340cc903d6 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:37.638104 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 998/1988] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:37.642673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3388b-9693-4b14-84e3-1b99372690f1 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:37.653502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3388b-9693-4b14-84e3-1b99372690f1 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:37.660645 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390aecc-37b7-4b23-bf29-0087b40f907b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:37.661325 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390aecc-37b7-4b23-bf29-0087b40f907b admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:37.661614 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390aecc-37b7-4b23-bf29-0087b40f907b admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:37.661938 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390aecc-37b7-4b23-bf29-0087b40f907b admin admin] PATH_INFO: `/v3/projects/9541b123f43442018e13ad1b6d577808/users/742f0a32d8de4dd2ba9915d07fd67f02/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:37.679909 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9390aecc-37b7-4b23-bf29-0087b40f907b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9541b123f43442018e13ad1b6d577808, user_id=742f0a32d8de4dd2ba9915d07fd67f02, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:37.682606 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9390aecc-37b7-4b23-bf29-0087b40f907b admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:37.698904 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 991/1989] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:37 2022] PUT /identity/v3/projects/9541b123f43442018e13ad1b6d577808/users/742f0a32d8de4dd2ba9915d07fd67f02/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:37.719130 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de7481-c15e-4bc2-b7cd-2905b4f20e91 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:37.719349 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de7481-c15e-4bc2-b7cd-2905b4f20e91 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:37.719628 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de7481-c15e-4bc2-b7cd-2905b4f20e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:37.752018 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-e4de7481-c15e-4bc2-b7cd-2905b4f20e91 None None] MFA Rules not processed for user `742f0a32d8de4dd2ba9915d07fd67f02`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:37.753486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-e4de7481-c15e-4bc2-b7cd-2905b4f20e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:37.776790 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390aecc-37b7-4b23-bf29-0087b40f907b admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:37.778674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9390aecc-37b7-4b23-bf29-0087b40f907b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:37.850019 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 999/1990] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:37.868898 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de7481-c15e-4bc2-b7cd-2905b4f20e91 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:37.871106 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-e4de7481-c15e-4bc2-b7cd-2905b4f20e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:37.879873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:37.940560 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390aecc-37b7-4b23-bf29-0087b40f907b admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:37.946013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95fa1d-d099-4190-92f9-629b1f01ce54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:37.946755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95fa1d-d099-4190-92f9-629b1f01ce54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:37.947009 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95fa1d-d099-4190-92f9-629b1f01ce54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:37.947351 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95fa1d-d099-4190-92f9-629b1f01ce54 admin admin] PATH_INFO: `/v3/projects/dc52b5d31e324b1ab41a3ee62d86c112` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:37.948973 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95fa1d-d099-4190-92f9-629b1f01ce54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc52b5d31e324b1ab41a3ee62d86c112)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:37.951650 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95fa1d-d099-4190-92f9-629b1f01ce54 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:37.965370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de7481-c15e-4bc2-b7cd-2905b4f20e91 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:37.970238 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a8b8c-ca18-471d-956b-341587277825 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:37.971063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a8b8c-ca18-471d-956b-341587277825 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:37.971455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a8b8c-ca18-471d-956b-341587277825 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:37.971902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a8b8c-ca18-471d-956b-341587277825 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:37.985074 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a8b8c-ca18-471d-956b-341587277825 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:37.986704 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a8b8c-ca18-471d-956b-341587277825 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:38.005902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1000/1991] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:13:38.040950 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-4a95fa1d-d099-4190-92f9-629b1f01ce54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc52b5d31e324b1ab41a3ee62d86c112', 'request_id': 'req-4a95fa1d-d099-4190-92f9-629b1f01ce54'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:38.042011 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a8b8c-ca18-471d-956b-341587277825 service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:38.054611 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-4a95fa1d-d099-4190-92f9-629b1f01ce54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc52b5d31e324b1ab41a3ee62d86c112', 'request_id': 'req-4a95fa1d-d099-4190-92f9-629b1f01ce54'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:38.062517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a8b8c-ca18-471d-956b-341587277825 service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:38.067203 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-4a95fa1d-d099-4190-92f9-629b1f01ce54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc52b5d31e324b1ab41a3ee62d86c112', 'request_id': 'req-4a95fa1d-d099-4190-92f9-629b1f01ce54'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:38.110279 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-4a95fa1d-d099-4190-92f9-629b1f01ce54 admin admin] The token cache is being invalidate because project dc52b5d31e324b1ab41a3ee62d86c112 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:13:38.111135 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-4a95fa1d-d099-4190-92f9-629b1f01ce54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:38.112244 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-4a95fa1d-d099-4190-92f9-629b1f01ce54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:38.117812 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 992/1992] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:13:37 2022] DELETE /identity/v3/projects/dc52b5d31e324b1ab41a3ee62d86c112 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:38.121629 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95fa1d-d099-4190-92f9-629b1f01ce54 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:38.124995 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-4a95fa1d-d099-4190-92f9-629b1f01ce54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:38.178790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a687a1-04d9-4d35-9ea1-5a38bee56c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:38.179726 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a687a1-04d9-4d35-9ea1-5a38bee56c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:38.180155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a687a1-04d9-4d35-9ea1-5a38bee56c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:38.180640 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a687a1-04d9-4d35-9ea1-5a38bee56c9b admin admin] PATH_INFO: `/v3/users/742f0a32d8de4dd2ba9915d07fd67f02` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:38.182403 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a687a1-04d9-4d35-9ea1-5a38bee56c9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=742f0a32d8de4dd2ba9915d07fd67f02)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:38.186247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a687a1-04d9-4d35-9ea1-5a38bee56c9b admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:38.207337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:INSERT=3,keystone:SELECT=96,keystone:DELETE=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:38.217245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95fa1d-d099-4190-92f9-629b1f01ce54 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:38.226277 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd0e28-2181-4806-af5c-311c5b123c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:38.227737 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd0e28-2181-4806-af5c-311c5b123c4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:38.227737 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd0e28-2181-4806-af5c-311c5b123c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:38.228126 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd0e28-2181-4806-af5c-311c5b123c4d admin admin] PATH_INFO: `/v3/users/2175699ac5564d8c845cdfb1cc2f065f` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:38.230104 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd0e28-2181-4806-af5c-311c5b123c4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2175699ac5564d8c845cdfb1cc2f065f)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:38.234386 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd0e28-2181-4806-af5c-311c5b123c4d admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:38.282497 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-60a687a1-04d9-4d35-9ea1-5a38bee56c9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '742f0a32d8de4dd2ba9915d07fd67f02'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:38.316326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-60a687a1-04d9-4d35-9ea1-5a38bee56c9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '742f0a32d8de4dd2ba9915d07fd67f02'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:38.336417 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-60a687a1-04d9-4d35-9ea1-5a38bee56c9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '742f0a32d8de4dd2ba9915d07fd67f02'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:38.345647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-e3fd0e28-2181-4806-af5c-311c5b123c4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2175699ac5564d8c845cdfb1cc2f065f'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:38.356352 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-60a687a1-04d9-4d35-9ea1-5a38bee56c9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '742f0a32d8de4dd2ba9915d07fd67f02'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:38.358537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-60a687a1-04d9-4d35-9ea1-5a38bee56c9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '742f0a32d8de4dd2ba9915d07fd67f02'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:38.359948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-e3fd0e28-2181-4806-af5c-311c5b123c4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2175699ac5564d8c845cdfb1cc2f065f'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:38.370196 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:38.380840 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1001/1993] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:13:38 2022] DELETE /identity/v3/users/742f0a32d8de4dd2ba9915d07fd67f02 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:13:38.390106 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9dce24-d1c5-4831-b84c-48eb6c941825 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:38.391035 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9dce24-d1c5-4831-b84c-48eb6c941825 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:38.392235 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9dce24-d1c5-4831-b84c-48eb6c941825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:38.399058 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-e3fd0e28-2181-4806-af5c-311c5b123c4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2175699ac5564d8c845cdfb1cc2f065f'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:38.415175 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-e3fd0e28-2181-4806-af5c-311c5b123c4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2175699ac5564d8c845cdfb1cc2f065f'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:38.416505 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-e3fd0e28-2181-4806-af5c-311c5b123c4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2175699ac5564d8c845cdfb1cc2f065f'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:38.430296 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 993/1994] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:13:38 2022] DELETE /identity/v3/users/2175699ac5564d8c845cdfb1cc2f065f => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:13:38.439218 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-4d9dce24-d1c5-4831-b84c-48eb6c941825 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:38.441801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-4d9dce24-d1c5-4831-b84c-48eb6c941825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:38.597930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1002/1995] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:13:38.607971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd0e28-2181-4806-af5c-311c5b123c4d admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:38.610674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-e3fd0e28-2181-4806-af5c-311c5b123c4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:38.723211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd0e28-2181-4806-af5c-311c5b123c4d admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:38.728894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66409e-9f08-43b0-a3eb-80bc6b75a187 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:38.730666 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66409e-9f08-43b0-a3eb-80bc6b75a187 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:38.731325 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66409e-9f08-43b0-a3eb-80bc6b75a187 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:38.731630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66409e-9f08-43b0-a3eb-80bc6b75a187 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:38.753236 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e66409e-9f08-43b0-a3eb-80bc6b75a187 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:38.755228 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e66409e-9f08-43b0-a3eb-80bc6b75a187 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:38.773411 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 994/1996] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:13:39.009393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9dce24-d1c5-4831-b84c-48eb6c941825 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:39.012392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-4d9dce24-d1c5-4831-b84c-48eb6c941825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:39.040499 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9dce24-d1c5-4831-b84c-48eb6c941825 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:39.046695 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641c729-3463-4dcd-a390-5f7b45d7edf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:39.047530 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641c729-3463-4dcd-a390-5f7b45d7edf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:39.056485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641c729-3463-4dcd-a390-5f7b45d7edf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:39.056881 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641c729-3463-4dcd-a390-5f7b45d7edf4 admin admin] PATH_INFO: `/v3/projects/d948be285b1844438c2c583d62b95f8f` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:39.058292 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641c729-3463-4dcd-a390-5f7b45d7edf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d948be285b1844438c2c583d62b95f8f)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:39.061173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641c729-3463-4dcd-a390-5f7b45d7edf4 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:39.172125 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-9641c729-3463-4dcd-a390-5f7b45d7edf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd948be285b1844438c2c583d62b95f8f', 'request_id': 'req-9641c729-3463-4dcd-a390-5f7b45d7edf4'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:39.182501 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-9641c729-3463-4dcd-a390-5f7b45d7edf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd948be285b1844438c2c583d62b95f8f', 'request_id': 'req-9641c729-3463-4dcd-a390-5f7b45d7edf4'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:39.194988 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-9641c729-3463-4dcd-a390-5f7b45d7edf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd948be285b1844438c2c583d62b95f8f', 'request_id': 'req-9641c729-3463-4dcd-a390-5f7b45d7edf4'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:39.217527 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-9641c729-3463-4dcd-a390-5f7b45d7edf4 admin admin] The token cache is being invalidate because project d948be285b1844438c2c583d62b95f8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:13:39.218187 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-9641c729-3463-4dcd-a390-5f7b45d7edf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:39.219213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-9641c729-3463-4dcd-a390-5f7b45d7edf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:39.222165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1003/1997] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:13:39 2022] DELETE /identity/v3/projects/d948be285b1844438c2c583d62b95f8f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:39.226865 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8106b9f-a209-4e4b-a36a-a208b9f4858c None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:39.227347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8106b9f-a209-4e4b-a36a-a208b9f4858c None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:39.227605 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8106b9f-a209-4e4b-a36a-a208b9f4858c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:39.275768 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-a8106b9f-a209-4e4b-a36a-a208b9f4858c None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:39.277998 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-a8106b9f-a209-4e4b-a36a-a208b9f4858c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:39.324477 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641c729-3463-4dcd-a390-5f7b45d7edf4 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:39.326583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-9641c729-3463-4dcd-a390-5f7b45d7edf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:39.361545 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:39.414831 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 995/1998] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:39.424147 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4acff-b3e6-4f58-85b8-2f09ddc18ea2 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:39.424437 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4acff-b3e6-4f58-85b8-2f09ddc18ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:39.424716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4acff-b3e6-4f58-85b8-2f09ddc18ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:39.448418 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-36e4acff-b3e6-4f58-85b8-2f09ddc18ea2 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:39.449770 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-36e4acff-b3e6-4f58-85b8-2f09ddc18ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:39.469818 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 996/1999] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:13:39.471492 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4acff-b3e6-4f58-85b8-2f09ddc18ea2 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:39.474207 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641c729-3463-4dcd-a390-5f7b45d7edf4 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:39.482092 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fffe2a-16a2-49da-8db0-894468a68146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:39.482719 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fffe2a-16a2-49da-8db0-894468a68146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:39.484315 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fffe2a-16a2-49da-8db0-894468a68146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:39.485486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fffe2a-16a2-49da-8db0-894468a68146 admin admin] PATH_INFO: `/v3/projects/9541b123f43442018e13ad1b6d577808` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:39.487255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fffe2a-16a2-49da-8db0-894468a68146 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9541b123f43442018e13ad1b6d577808)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:39.488936 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4acff-b3e6-4f58-85b8-2f09ddc18ea2 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:39.489405 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fffe2a-16a2-49da-8db0-894468a68146 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:39.511253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ecc3f-1362-4f66-8512-fb2a17e1c013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:39.512022 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ecc3f-1362-4f66-8512-fb2a17e1c013 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:39.512400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ecc3f-1362-4f66-8512-fb2a17e1c013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:39.512820 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ecc3f-1362-4f66-8512-fb2a17e1c013 admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:39.513866 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ecc3f-1362-4f66-8512-fb2a17e1c013 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:13:39.514304 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ecc3f-1362-4f66-8512-fb2a17e1c013 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:39.516029 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ecc3f-1362-4f66-8512-fb2a17e1c013 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:39.538872 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 997/2000] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:13:39.551500 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ecc3f-1362-4f66-8512-fb2a17e1c013 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:39.580790 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ecc3f-1362-4f66-8512-fb2a17e1c013 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:39.581094 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-42fffe2a-16a2-49da-8db0-894468a68146 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9541b123f43442018e13ad1b6d577808', 'request_id': 'req-42fffe2a-16a2-49da-8db0-894468a68146'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:39.590978 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-42fffe2a-16a2-49da-8db0-894468a68146 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9541b123f43442018e13ad1b6d577808', 'request_id': 'req-42fffe2a-16a2-49da-8db0-894468a68146'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:39.601306 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-42fffe2a-16a2-49da-8db0-894468a68146 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9541b123f43442018e13ad1b6d577808', 'request_id': 'req-42fffe2a-16a2-49da-8db0-894468a68146'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:39.679153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-42fffe2a-16a2-49da-8db0-894468a68146 admin admin] The token cache is being invalidate because project 9541b123f43442018e13ad1b6d577808 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:13:39.679397 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-42fffe2a-16a2-49da-8db0-894468a68146 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:39.684844 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-42fffe2a-16a2-49da-8db0-894468a68146 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:39.691955 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1004/2001] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:13:39 2022] DELETE /identity/v3/projects/9541b123f43442018e13ad1b6d577808 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:13:39.701523 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d842de4-4e1e-4496-9c81-f97b156f7936 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:39.701845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d842de4-4e1e-4496-9c81-f97b156f7936 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:39.702134 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d842de4-4e1e-4496-9c81-f97b156f7936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:39.737137 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb812e8-2ee1-4cdc-aa0b-2513b638ea0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:39.737813 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb812e8-2ee1-4cdc-aa0b-2513b638ea0e admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:39.738104 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb812e8-2ee1-4cdc-aa0b-2513b638ea0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:39.738382 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb812e8-2ee1-4cdc-aa0b-2513b638ea0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:39.739074 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb812e8-2ee1-4cdc-aa0b-2513b638ea0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:39.741203 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb812e8-2ee1-4cdc-aa0b-2513b638ea0e admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:39.749511 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:39.767612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-1d842de4-4e1e-4496-9c81-f97b156f7936 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:39.769828 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-1d842de4-4e1e-4496-9c81-f97b156f7936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:39.799662 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1005/2002] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:39.802773 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210dfdb-297a-4bae-9637-27b16a9829ba None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:39.803998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210dfdb-297a-4bae-9637-27b16a9829ba None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:39.803998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210dfdb-297a-4bae-9637-27b16a9829ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:39.809868 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 998/2003] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:13:39 2022] POST /identity/v3/projects => generated 385 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:39.814887 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb812e8-2ee1-4cdc-aa0b-2513b638ea0e admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:39.818007 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-3eb812e8-2ee1-4cdc-aa0b-2513b638ea0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:39.864397 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-d210dfdb-297a-4bae-9637-27b16a9829ba None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:39.866825 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-d210dfdb-297a-4bae-9637-27b16a9829ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:39.946662 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb812e8-2ee1-4cdc-aa0b-2513b638ea0e admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:39.966202 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43616cfe-694d-4a98-86cb-83baa963e465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:39.967738 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43616cfe-694d-4a98-86cb-83baa963e465 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:39.968208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43616cfe-694d-4a98-86cb-83baa963e465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:39.970136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43616cfe-694d-4a98-86cb-83baa963e465 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:39.971536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43616cfe-694d-4a98-86cb-83baa963e465 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:39.978560 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43616cfe-694d-4a98-86cb-83baa963e465 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:40.013772 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 999/2004] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:13:39 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:40.019830 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43616cfe-694d-4a98-86cb-83baa963e465 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:40.028573 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1006/2005] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:40.031568 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210dfdb-297a-4bae-9637-27b16a9829ba None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:40.036069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43616cfe-694d-4a98-86cb-83baa963e465 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:40.041389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25873eb2-3e10-4694-8d44-ad0c31fb54ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:40.042047 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25873eb2-3e10-4694-8d44-ad0c31fb54ce admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:40.042332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25873eb2-3e10-4694-8d44-ad0c31fb54ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:40.042612 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25873eb2-3e10-4694-8d44-ad0c31fb54ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:40.043355 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25873eb2-3e10-4694-8d44-ad0c31fb54ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:40.045909 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25873eb2-3e10-4694-8d44-ad0c31fb54ce admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:40.050911 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210dfdb-297a-4bae-9637-27b16a9829ba None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:40.061839 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1000/2006] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:13:40.063627 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ec657-00da-46fe-9a99-3028025db0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:40.064101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25873eb2-3e10-4694-8d44-ad0c31fb54ce admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:40.065052 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ec657-00da-46fe-9a99-3028025db0af admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:40.066075 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ec657-00da-46fe-9a99-3028025db0af admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:40.066708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ec657-00da-46fe-9a99-3028025db0af admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:40.068608 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ec657-00da-46fe-9a99-3028025db0af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:13:40.068706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ec657-00da-46fe-9a99-3028025db0af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:40.094735 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ec657-00da-46fe-9a99-3028025db0af admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:40.099510 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25873eb2-3e10-4694-8d44-ad0c31fb54ce admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:40.124174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644c49f-3ed9-4100-95e7-723b0e2b6481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:40.125291 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644c49f-3ed9-4100-95e7-723b0e2b6481 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:40.127617 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644c49f-3ed9-4100-95e7-723b0e2b6481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:40.129505 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644c49f-3ed9-4100-95e7-723b0e2b6481 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:40.132288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b644c49f-3ed9-4100-95e7-723b0e2b6481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:40.138243 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1007/2007] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:13:40.138431 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b644c49f-3ed9-4100-95e7-723b0e2b6481 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:40.144963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ec657-00da-46fe-9a99-3028025db0af admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:40.158715 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1001/2008] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:40.164276 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644c49f-3ed9-4100-95e7-723b0e2b6481 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:40.165355 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ec657-00da-46fe-9a99-3028025db0af admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:40.174155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9386b0-40f9-4f61-a149-d375f228898a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:40.175403 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9386b0-40f9-4f61-a149-d375f228898a admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:40.175875 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9386b0-40f9-4f61-a149-d375f228898a admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:40.176339 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9386b0-40f9-4f61-a149-d375f228898a admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:40.177298 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644c49f-3ed9-4100-95e7-723b0e2b6481 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:40.177487 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9386b0-40f9-4f61-a149-d375f228898a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:40.181194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9386b0-40f9-4f61-a149-d375f228898a admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:40.183608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de00f5-4dc0-4975-b0fb-933479bd6480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:40.183608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de00f5-4dc0-4975-b0fb-933479bd6480 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:40.184016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de00f5-4dc0-4975-b0fb-933479bd6480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:40.184360 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de00f5-4dc0-4975-b0fb-933479bd6480 admin admin] PATH_INFO: `/v3/projects/146824c806c14bd2b68b149bda56e2b8/users/cfebe82d1e014ca8bf0dc7c5f26c6045/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:40.209261 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de00f5-4dc0-4975-b0fb-933479bd6480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=146824c806c14bd2b68b149bda56e2b8, user_id=cfebe82d1e014ca8bf0dc7c5f26c6045, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:40.212746 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de00f5-4dc0-4975-b0fb-933479bd6480 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:40.231574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1008/2009] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:13:40 2022] POST /identity/v3/projects => generated 385 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:13:40.233082 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1002/2010] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:40 2022] PUT /identity/v3/projects/146824c806c14bd2b68b149bda56e2b8/users/cfebe82d1e014ca8bf0dc7c5f26c6045/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:40.238657 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9386b0-40f9-4f61-a149-d375f228898a admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:40.245075 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:40.255399 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0994c6-e9ac-4421-a950-2b3e6cf5f7e4 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:40.256182 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0994c6-e9ac-4421-a950-2b3e6cf5f7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:40.256662 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0994c6-e9ac-4421-a950-2b3e6cf5f7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:40.301862 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-7d0994c6-e9ac-4421-a950-2b3e6cf5f7e4 None None] MFA Rules not processed for user `cfebe82d1e014ca8bf0dc7c5f26c6045`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:40.303239 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-7d0994c6-e9ac-4421-a950-2b3e6cf5f7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:40.394809 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9386b0-40f9-4f61-a149-d375f228898a admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:40.414972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27683ee6-6169-4efa-94e8-5d5335c9353f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:40.417068 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27683ee6-6169-4efa-94e8-5d5335c9353f admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:40.418341 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27683ee6-6169-4efa-94e8-5d5335c9353f admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:40.419033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27683ee6-6169-4efa-94e8-5d5335c9353f admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:40.421321 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27683ee6-6169-4efa-94e8-5d5335c9353f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:40.424762 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27683ee6-6169-4efa-94e8-5d5335c9353f admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:40.456409 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1003/2011] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:40.463723 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0994c6-e9ac-4421-a950-2b3e6cf5f7e4 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:40.474632 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1009/2012] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:13:40 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:40.483641 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27683ee6-6169-4efa-94e8-5d5335c9353f admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:40.485465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0994c6-e9ac-4421-a950-2b3e6cf5f7e4 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:40.491352 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9e8d1-6d16-401b-9c15-de97ea865e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:40.492067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9e8d1-6d16-401b-9c15-de97ea865e1e admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:40.492414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9e8d1-6d16-401b-9c15-de97ea865e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:40.493757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9e8d1-6d16-401b-9c15-de97ea865e1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:40.493757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9e8d1-6d16-401b-9c15-de97ea865e1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:40.495599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9e8d1-6d16-401b-9c15-de97ea865e1e admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:40.515547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27683ee6-6169-4efa-94e8-5d5335c9353f admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:40.524700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7b5b5-951f-4af7-a646-27766c1a52a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:40.525605 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7b5b5-951f-4af7-a646-27766c1a52a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:40.526229 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7b5b5-951f-4af7-a646-27766c1a52a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:40.526753 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7b5b5-951f-4af7-a646-27766c1a52a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:40.527906 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7b5b5-951f-4af7-a646-27766c1a52a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:40.534272 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7b5b5-951f-4af7-a646-27766c1a52a4 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:40.545384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:40.555109 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1010/2013] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:40.558599 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7b5b5-951f-4af7-a646-27766c1a52a4 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:40.560782 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1004/2014] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:13:40 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:40.567904 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9e8d1-6d16-401b-9c15-de97ea865e1e admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:40.580433 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7b5b5-951f-4af7-a646-27766c1a52a4 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:40.665281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1494a8e3-a72b-4436-811e-1c3748c7d8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:40.666349 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494a8e3-a72b-4436-811e-1c3748c7d8a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:40.666632 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9e8d1-6d16-401b-9c15-de97ea865e1e admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:40.666832 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494a8e3-a72b-4436-811e-1c3748c7d8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:40.667242 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494a8e3-a72b-4436-811e-1c3748c7d8a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:40.668184 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1494a8e3-a72b-4436-811e-1c3748c7d8a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:40.670559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1494a8e3-a72b-4436-811e-1c3748c7d8a1 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:40.675579 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff7952-94fe-4b67-baa3-497227ebdd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:40.677363 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff7952-94fe-4b67-baa3-497227ebdd50 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:40.677363 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff7952-94fe-4b67-baa3-497227ebdd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:40.677679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff7952-94fe-4b67-baa3-497227ebdd50 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:40.678745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff7952-94fe-4b67-baa3-497227ebdd50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:40.682858 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff7952-94fe-4b67-baa3-497227ebdd50 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:40.685990 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1011/2015] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:13:40.693873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1494a8e3-a72b-4436-811e-1c3748c7d8a1 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:40.716975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1494a8e3-a72b-4436-811e-1c3748c7d8a1 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:40.727096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad384e30-0bea-4fec-bc89-3bda9b3316d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:40.728453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad384e30-0bea-4fec-bc89-3bda9b3316d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:40.729028 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad384e30-0bea-4fec-bc89-3bda9b3316d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:40.729759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad384e30-0bea-4fec-bc89-3bda9b3316d0 admin admin] PATH_INFO: `/v3/projects/6f804787598449208b834ef89468713f/users/67b6369b61ed4c6c93b55f199b728210/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:40.732947 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1005/2016] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:13:40 2022] POST /identity/v3/users => generated 299 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:13:40.742191 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff7952-94fe-4b67-baa3-497227ebdd50 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:40.763315 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff7952-94fe-4b67-baa3-497227ebdd50 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:40.773254 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5bc98-7563-43a9-8d6c-a4d81064f9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:40.773900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad384e30-0bea-4fec-bc89-3bda9b3316d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f804787598449208b834ef89468713f, user_id=67b6369b61ed4c6c93b55f199b728210, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:40.774466 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5bc98-7563-43a9-8d6c-a4d81064f9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:40.774954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5bc98-7563-43a9-8d6c-a4d81064f9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:40.775616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5bc98-7563-43a9-8d6c-a4d81064f9c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:40.777431 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5bc98-7563-43a9-8d6c-a4d81064f9c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:40.780019 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5bc98-7563-43a9-8d6c-a4d81064f9c8 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:40.784165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad384e30-0bea-4fec-bc89-3bda9b3316d0 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:40.808519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1006/2017] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:40.812803 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5bc98-7563-43a9-8d6c-a4d81064f9c8 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:40.813896 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1012/2018] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:40 2022] PUT /identity/v3/projects/6f804787598449208b834ef89468713f/users/67b6369b61ed4c6c93b55f199b728210/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:13:40.824092 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5bc98-7563-43a9-8d6c-a4d81064f9c8 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:40.827370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000bfdb-e061-424c-855e-da3340bcefd2 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:40.827596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000bfdb-e061-424c-855e-da3340bcefd2 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:40.827792 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000bfdb-e061-424c-855e-da3340bcefd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:40.864201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-8000bfdb-e061-424c-855e-da3340bcefd2 None None] MFA Rules not processed for user `67b6369b61ed4c6c93b55f199b728210`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:40.865822 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-8000bfdb-e061-424c-855e-da3340bcefd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:40.931618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:40.950854 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb664c3e-c435-4499-bf6e-f005a9249a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:40.952082 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb664c3e-c435-4499-bf6e-f005a9249a54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:40.952369 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb664c3e-c435-4499-bf6e-f005a9249a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:40.952679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb664c3e-c435-4499-bf6e-f005a9249a54 admin admin] PATH_INFO: `/v3/projects/1727f4ea13024749b0266b53777ea8cd/users/62a0003f31484004a6bff502910f066d/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:40.999184 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1013/2019] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:40.999184 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7a349-6107-48e7-8770-0b8794a84cc9 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:40.999184 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7a349-6107-48e7-8770-0b8794a84cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:40.999184 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7a349-6107-48e7-8770-0b8794a84cc9 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:41.001111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1014/2020] 10.222.0.73 () {54 vars in 921 bytes} [Mon Aug 29 20:13:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:13:41.001777 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb664c3e-c435-4499-bf6e-f005a9249a54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1727f4ea13024749b0266b53777ea8cd, user_id=62a0003f31484004a6bff502910f066d, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:41.007441 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5be19-6b0c-4628-a7c3-78ac9dc0571a None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:41.007669 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb664c3e-c435-4499-bf6e-f005a9249a54 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:41.008380 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5be19-6b0c-4628-a7c3-78ac9dc0571a None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:41.008762 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5be19-6b0c-4628-a7c3-78ac9dc0571a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:41.049060 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1007/2021] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:40 2022] PUT /identity/v3/projects/1727f4ea13024749b0266b53777ea8cd/users/62a0003f31484004a6bff502910f066d/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:13:41.065017 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb664c3e-c435-4499-bf6e-f005a9249a54 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:41.080567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-ddd5be19-6b0c-4628-a7c3-78ac9dc0571a None None] MFA Rules not processed for user `1ab2e318b2854c46bc7ed2151a44a2e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:41.103303 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ddd5be19-6b0c-4628-a7c3-78ac9dc0571a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:41.249765 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb664c3e-c435-4499-bf6e-f005a9249a54 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:41.260601 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76a2cc-e7e6-48c6-a820-0f10fc8bb153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:41.261744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76a2cc-e7e6-48c6-a820-0f10fc8bb153 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:41.262258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76a2cc-e7e6-48c6-a820-0f10fc8bb153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:41.263709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76a2cc-e7e6-48c6-a820-0f10fc8bb153 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:41.265013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76a2cc-e7e6-48c6-a820-0f10fc8bb153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:41.268537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1015/2022] 10.222.0.73 () {58 vars in 1018 bytes} [Mon Aug 29 20:13:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:13:41.269039 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76a2cc-e7e6-48c6-a820-0f10fc8bb153 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:41.270260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5be19-6b0c-4628-a7c3-78ac9dc0571a None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:41.272540 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:41.272726 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ddd5be19-6b0c-4628-a7c3-78ac9dc0571a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:41.300354 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1008/2023] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:41.301963 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76a2cc-e7e6-48c6-a820-0f10fc8bb153 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:41.321348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76a2cc-e7e6-48c6-a820-0f10fc8bb153 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:41.336255 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573505d-f968-4525-9493-e7c2616cc8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:41.337348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573505d-f968-4525-9493-e7c2616cc8ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:41.338020 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573505d-f968-4525-9493-e7c2616cc8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:41.338470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573505d-f968-4525-9493-e7c2616cc8ba admin admin] PATH_INFO: `/v3/projects/1727f4ea13024749b0266b53777ea8cd/users/62a0003f31484004a6bff502910f066d/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:41.342393 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5573505d-f968-4525-9493-e7c2616cc8ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=1727f4ea13024749b0266b53777ea8cd, user_id=62a0003f31484004a6bff502910f066d, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:41.347714 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5573505d-f968-4525-9493-e7c2616cc8ba admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:41.365095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5be19-6b0c-4628-a7c3-78ac9dc0571a None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:41.372299 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1009/2024] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:41 2022] PUT /identity/v3/projects/1727f4ea13024749b0266b53777ea8cd/users/62a0003f31484004a6bff502910f066d/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:41.374312 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2d073-b451-4c51-88a2-d2b4b94a5720 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:41.374611 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2d073-b451-4c51-88a2-d2b4b94a5720 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:41.374890 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2d073-b451-4c51-88a2-d2b4b94a5720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:41.418965 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-1ce2d073-b451-4c51-88a2-d2b4b94a5720 None None] MFA Rules not processed for user `62a0003f31484004a6bff502910f066d`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:41.420749 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-1ce2d073-b451-4c51-88a2-d2b4b94a5720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:41.469801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74062c4e-bc66-4b3b-92ef-ca781c6fb41c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:41.472256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74062c4e-bc66-4b3b-92ef-ca781c6fb41c service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:41.474420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74062c4e-bc66-4b3b-92ef-ca781c6fb41c service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:41.476788 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74062c4e-bc66-4b3b-92ef-ca781c6fb41c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:41.559324 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:41.574116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74062c4e-bc66-4b3b-92ef-ca781c6fb41c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:41.576797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74062c4e-bc66-4b3b-92ef-ca781c6fb41c service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:41.591994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1010/2025] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:41.596126 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71073833-1ac5-4014-8158-61b89ac7816a None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:41.596417 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71073833-1ac5-4014-8158-61b89ac7816a None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:41.596690 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71073833-1ac5-4014-8158-61b89ac7816a None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:41.598384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1016/2026] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:13:41.599400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1011/2027] 10.222.0.73 () {54 vars in 961 bytes} [Mon Aug 29 20:13:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:13:41.610526 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e301d-397a-4a4f-a134-925b1051afc6 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:41.610999 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e301d-397a-4a4f-a134-925b1051afc6 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:41.611449 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e301d-397a-4a4f-a134-925b1051afc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:41.649356 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-764e301d-397a-4a4f-a134-925b1051afc6 None None] MFA Rules not processed for user `1ab2e318b2854c46bc7ed2151a44a2e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:41.651669 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-764e301d-397a-4a4f-a134-925b1051afc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:41.790347 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1017/2028] 10.222.0.73 () {58 vars in 1058 bytes} [Mon Aug 29 20:13:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:41.799999 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71073833-1ac5-4014-8158-61b89ac7816a None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:41.837366 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71073833-1ac5-4014-8158-61b89ac7816a None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:41.848571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d8b69-c8ea-4819-a828-1e5190524a81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab2e318b2854c46bc7ed2151a44a2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:41.849415 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d8b69-c8ea-4819-a828-1e5190524a81 service placement] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:41.850021 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d8b69-c8ea-4819-a828-1e5190524a81 service placement] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:41.850444 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d8b69-c8ea-4819-a828-1e5190524a81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:41.873789 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911d8b69-c8ea-4819-a828-1e5190524a81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:41.878530 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911d8b69-c8ea-4819-a828-1e5190524a81 service placement] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:41.909900 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1012/2029] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:13:42.104984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e301d-397a-4a4f-a134-925b1051afc6 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:42.146685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e301d-397a-4a4f-a134-925b1051afc6 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:42.155388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d2338-c028-4dc8-873d-fbcd57fb3729 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:42.156554 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d2338-c028-4dc8-873d-fbcd57fb3729 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:42.157108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d2338-c028-4dc8-873d-fbcd57fb3729 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:42.157665 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d2338-c028-4dc8-873d-fbcd57fb3729 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:42.247356 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d8b69-c8ea-4819-a828-1e5190524a81 service placement] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:42.250209 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-911d8b69-c8ea-4819-a828-1e5190524a81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:42.401753 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d2338-c028-4dc8-873d-fbcd57fb3729 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:42.404923 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d2338-c028-4dc8-873d-fbcd57fb3729 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:42.459811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1018/2030] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:13:42.494676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d8b69-c8ea-4819-a828-1e5190524a81 service placement] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:42.508160 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657aa5e-983e-4123-b2b9-e62a14a6e25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:42.509261 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657aa5e-983e-4123-b2b9-e62a14a6e25a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:42.509973 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657aa5e-983e-4123-b2b9-e62a14a6e25a admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:42.511267 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657aa5e-983e-4123-b2b9-e62a14a6e25a admin admin] PATH_INFO: `/v3/users/93dda2d3d8bd4d2ca4b80dc9a09fcf81` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:42.516409 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657aa5e-983e-4123-b2b9-e62a14a6e25a admin admin] RBAC: Authorizing `identity:delete_user(user_id=93dda2d3d8bd4d2ca4b80dc9a09fcf81)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:42.520573 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657aa5e-983e-4123-b2b9-e62a14a6e25a admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:42.632397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b657aa5e-983e-4123-b2b9-e62a14a6e25a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93dda2d3d8bd4d2ca4b80dc9a09fcf81'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:42.651138 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b657aa5e-983e-4123-b2b9-e62a14a6e25a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93dda2d3d8bd4d2ca4b80dc9a09fcf81'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:42.654687 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:42.694220 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b657aa5e-983e-4123-b2b9-e62a14a6e25a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93dda2d3d8bd4d2ca4b80dc9a09fcf81'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:42.715191 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b657aa5e-983e-4123-b2b9-e62a14a6e25a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93dda2d3d8bd4d2ca4b80dc9a09fcf81'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:42.720277 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b657aa5e-983e-4123-b2b9-e62a14a6e25a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93dda2d3d8bd4d2ca4b80dc9a09fcf81'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:42.736346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1013/2031] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:13:42 2022] DELETE /identity/v3/users/93dda2d3d8bd4d2ca4b80dc9a09fcf81 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:13:43.040166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d2338-c028-4dc8-873d-fbcd57fb3729 service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:43.074984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d2338-c028-4dc8-873d-fbcd57fb3729 service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:43.249910 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bea9ce-6b20-46d2-9473-cef6cc2f51b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:43.250717 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bea9ce-6b20-46d2-9473-cef6cc2f51b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:43.251012 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bea9ce-6b20-46d2-9473-cef6cc2f51b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:43.251291 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bea9ce-6b20-46d2-9473-cef6cc2f51b3 admin admin] PATH_INFO: `/v3/projects/c5163a9611df47069421fe45e55c7ab4` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:43.253954 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bea9ce-6b20-46d2-9473-cef6cc2f51b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5163a9611df47069421fe45e55c7ab4)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:43.256372 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bea9ce-6b20-46d2-9473-cef6cc2f51b3 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:43.316931 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:43.350676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a8bea9ce-6b20-46d2-9473-cef6cc2f51b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5163a9611df47069421fe45e55c7ab4', 'request_id': 'req-a8bea9ce-6b20-46d2-9473-cef6cc2f51b3'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:43.360315 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a8bea9ce-6b20-46d2-9473-cef6cc2f51b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5163a9611df47069421fe45e55c7ab4', 'request_id': 'req-a8bea9ce-6b20-46d2-9473-cef6cc2f51b3'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:43.374564 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a8bea9ce-6b20-46d2-9473-cef6cc2f51b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5163a9611df47069421fe45e55c7ab4', 'request_id': 'req-a8bea9ce-6b20-46d2-9473-cef6cc2f51b3'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:43.401428 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a8bea9ce-6b20-46d2-9473-cef6cc2f51b3 admin admin] The token cache is being invalidate because project c5163a9611df47069421fe45e55c7ab4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:13:43.401930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a8bea9ce-6b20-46d2-9473-cef6cc2f51b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:43.402760 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a8bea9ce-6b20-46d2-9473-cef6cc2f51b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:43.409515 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1019/2032] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:13:43 2022] DELETE /identity/v3/projects/c5163a9611df47069421fe45e55c7ab4 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:43.412356 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657aa5e-983e-4123-b2b9-e62a14a6e25a admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:43.414320 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-b657aa5e-983e-4123-b2b9-e62a14a6e25a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:43.440904 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657aa5e-983e-4123-b2b9-e62a14a6e25a admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:43.572877 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a0c3a-7536-485e-936f-aeabed678862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:43.574524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a0c3a-7536-485e-936f-aeabed678862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:43.574524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a0c3a-7536-485e-936f-aeabed678862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:43.574928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a0c3a-7536-485e-936f-aeabed678862 admin admin] PATH_INFO: `/v3/users/2a7822281eed4dc4b103f25e30bcc48a` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:43.577973 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a0c3a-7536-485e-936f-aeabed678862 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a7822281eed4dc4b103f25e30bcc48a)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:43.583708 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a0c3a-7536-485e-936f-aeabed678862 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:43.677512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-062a0c3a-7536-485e-936f-aeabed678862 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a7822281eed4dc4b103f25e30bcc48a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:43.689345 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-062a0c3a-7536-485e-936f-aeabed678862 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a7822281eed4dc4b103f25e30bcc48a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:43.708044 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-062a0c3a-7536-485e-936f-aeabed678862 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a7822281eed4dc4b103f25e30bcc48a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:43.723745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-062a0c3a-7536-485e-936f-aeabed678862 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a7822281eed4dc4b103f25e30bcc48a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:43.725574 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-062a0c3a-7536-485e-936f-aeabed678862 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a7822281eed4dc4b103f25e30bcc48a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:43.738524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1014/2033] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:13:43 2022] DELETE /identity/v3/users/2a7822281eed4dc4b103f25e30bcc48a => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:44.034042 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bea9ce-6b20-46d2-9473-cef6cc2f51b3 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:44.035551 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a8bea9ce-6b20-46d2-9473-cef6cc2f51b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:44.056391 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bea9ce-6b20-46d2-9473-cef6cc2f51b3 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:44.170731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42335775-908a-45ec-a574-e47891c666f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:44.171309 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42335775-908a-45ec-a574-e47891c666f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:44.171591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42335775-908a-45ec-a574-e47891c666f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:44.171866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42335775-908a-45ec-a574-e47891c666f0 admin admin] PATH_INFO: `/v3/projects/23e5f9aab63542f99fda007487f5b109` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:44.173098 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42335775-908a-45ec-a574-e47891c666f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23e5f9aab63542f99fda007487f5b109)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:44.175102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42335775-908a-45ec-a574-e47891c666f0 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:44.256208 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-42335775-908a-45ec-a574-e47891c666f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23e5f9aab63542f99fda007487f5b109', 'request_id': 'req-42335775-908a-45ec-a574-e47891c666f0'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:44.264225 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-42335775-908a-45ec-a574-e47891c666f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23e5f9aab63542f99fda007487f5b109', 'request_id': 'req-42335775-908a-45ec-a574-e47891c666f0'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:44.282539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-42335775-908a-45ec-a574-e47891c666f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23e5f9aab63542f99fda007487f5b109', 'request_id': 'req-42335775-908a-45ec-a574-e47891c666f0'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:44.308117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-42335775-908a-45ec-a574-e47891c666f0 admin admin] The token cache is being invalidate because project 23e5f9aab63542f99fda007487f5b109 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:13:44.308751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-42335775-908a-45ec-a574-e47891c666f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:44.309857 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-42335775-908a-45ec-a574-e47891c666f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:44.313978 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1020/2034] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:13:44 2022] DELETE /identity/v3/projects/23e5f9aab63542f99fda007487f5b109 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:44.323854 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a9810-2558-404e-8b74-6869b4a87c5f None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:44.325668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a9810-2558-404e-8b74-6869b4a87c5f None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:44.326620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a9810-2558-404e-8b74-6869b4a87c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:44.365007 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-fc3a9810-2558-404e-8b74-6869b4a87c5f None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:44.367210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-fc3a9810-2558-404e-8b74-6869b4a87c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:44.433069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:44.450259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1015/2035] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:13:44.462976 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0e239-84b6-4c2e-b1e6-c4a3abdf6f1d None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:44.463616 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0e239-84b6-4c2e-b1e6-c4a3abdf6f1d None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:44.464198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0e239-84b6-4c2e-b1e6-c4a3abdf6f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:44.509134 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-b4b0e239-84b6-4c2e-b1e6-c4a3abdf6f1d None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:44.509134 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-b4b0e239-84b6-4c2e-b1e6-c4a3abdf6f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:44.546823 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1021/2036] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:44.552607 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a9810-2558-404e-8b74-6869b4a87c5f None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:44.569078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a9810-2558-404e-8b74-6869b4a87c5f None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:44.576915 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e9593-1549-4003-b924-3a18a9fb7c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:44.577854 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e9593-1549-4003-b924-3a18a9fb7c0c admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:44.578573 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e9593-1549-4003-b924-3a18a9fb7c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:44.579059 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e9593-1549-4003-b924-3a18a9fb7c0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:44.580140 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e9593-1549-4003-b924-3a18a9fb7c0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:13:44.580785 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e9593-1549-4003-b924-3a18a9fb7c0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:44.583520 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e9593-1549-4003-b924-3a18a9fb7c0c admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:44.600601 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1016/2037] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:13:44.606718 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0e239-84b6-4c2e-b1e6-c4a3abdf6f1d None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:44.618514 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0e239-84b6-4c2e-b1e6-c4a3abdf6f1d None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:44.640659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e8c3c-6d54-43a4-9699-c7623d73da86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:44.641303 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e8c3c-6d54-43a4-9699-c7623d73da86 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:44.641584 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e8c3c-6d54-43a4-9699-c7623d73da86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:44.641889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e8c3c-6d54-43a4-9699-c7623d73da86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:44.642556 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e8c3c-6d54-43a4-9699-c7623d73da86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:44.644473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e8c3c-6d54-43a4-9699-c7623d73da86 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:44.683435 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1022/2038] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:13:44 2022] POST /identity/v3/projects => generated 361 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:44.685408 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e9593-1549-4003-b924-3a18a9fb7c0c admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:44.699700 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e9593-1549-4003-b924-3a18a9fb7c0c admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:44.779601 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6e82c-d516-4406-84e1-0c709976fb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:44.780514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6e82c-d516-4406-84e1-0c709976fb25 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:44.780860 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6e82c-d516-4406-84e1-0c709976fb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:44.781184 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6e82c-d516-4406-84e1-0c709976fb25 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:44.781930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e6e82c-d516-4406-84e1-0c709976fb25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:44.783942 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e6e82c-d516-4406-84e1-0c709976fb25 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:44.810433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1017/2039] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:13:44 2022] POST /identity/v3/users => generated 281 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:44.814692 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e8c3c-6d54-43a4-9699-c7623d73da86 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:44.825251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e8c3c-6d54-43a4-9699-c7623d73da86 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:44.832016 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd68a33-1566-4fc8-a663-a9c5e671e1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:44.832947 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd68a33-1566-4fc8-a663-a9c5e671e1d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:44.833427 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd68a33-1566-4fc8-a663-a9c5e671e1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:44.833926 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd68a33-1566-4fc8-a663-a9c5e671e1d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:44.834638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd68a33-1566-4fc8-a663-a9c5e671e1d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:44.837229 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd68a33-1566-4fc8-a663-a9c5e671e1d2 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:44.851075 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1023/2040] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:44.857519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6e82c-d516-4406-84e1-0c709976fb25 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:44.870347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6e82c-d516-4406-84e1-0c709976fb25 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:44.877607 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524adffa-971b-492d-beea-974f3731d3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:44.878268 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524adffa-971b-492d-beea-974f3731d3b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:44.878545 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524adffa-971b-492d-beea-974f3731d3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:44.878825 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524adffa-971b-492d-beea-974f3731d3b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:44.879483 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524adffa-971b-492d-beea-974f3731d3b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:44.881673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524adffa-971b-492d-beea-974f3731d3b2 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:44.896407 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1018/2041] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:44.901685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd68a33-1566-4fc8-a663-a9c5e671e1d2 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:44.914234 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:44.921145 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd68a33-1566-4fc8-a663-a9c5e671e1d2 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:44.942293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21648f9-a12d-447f-ad49-9864299dea4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:44.943197 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21648f9-a12d-447f-ad49-9864299dea4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:44.943651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21648f9-a12d-447f-ad49-9864299dea4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:44.944102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21648f9-a12d-447f-ad49-9864299dea4c admin admin] PATH_INFO: `/v3/projects/5c4db85de834450293f4545755baae37/users/bdeed465ce2e481fa01e6c35f44fa7f6/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:44.976708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21648f9-a12d-447f-ad49-9864299dea4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c4db85de834450293f4545755baae37, user_id=bdeed465ce2e481fa01e6c35f44fa7f6, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:44.977873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21648f9-a12d-447f-ad49-9864299dea4c admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:44.998217 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1024/2042] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:44 2022] PUT /identity/v3/projects/5c4db85de834450293f4545755baae37/users/bdeed465ce2e481fa01e6c35f44fa7f6/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:13:45.012411 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f4a54-dc48-4f35-8311-6a47811b893a None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:45.012813 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f4a54-dc48-4f35-8311-6a47811b893a None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:45.013108 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f4a54-dc48-4f35-8311-6a47811b893a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:45.052789 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-8b1f4a54-dc48-4f35-8311-6a47811b893a None None] MFA Rules not processed for user `bdeed465ce2e481fa01e6c35f44fa7f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:45.054060 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-8b1f4a54-dc48-4f35-8311-6a47811b893a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:45.133294 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1019/2043] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:45.177163 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21648f9-a12d-447f-ad49-9864299dea4c admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:45.179009 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-b21648f9-a12d-447f-ad49-9864299dea4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:45.270083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21648f9-a12d-447f-ad49-9864299dea4c admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:45.276773 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1b527-06c7-47e5-8c68-22f19c6459b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:45.277814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1b527-06c7-47e5-8c68-22f19c6459b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:45.278452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1b527-06c7-47e5-8c68-22f19c6459b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:45.279369 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1b527-06c7-47e5-8c68-22f19c6459b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:45.298168 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d1b527-06c7-47e5-8c68-22f19c6459b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:45.300212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d1b527-06c7-47e5-8c68-22f19c6459b4 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:45.316754 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1025/2044] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:13:45.334921 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f4a54-dc48-4f35-8311-6a47811b893a None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:45.392124 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:45.412573 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f4a54-dc48-4f35-8311-6a47811b893a None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:45.420196 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf8adf-f2fa-45fd-957f-a1f179b0bac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:45.421764 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf8adf-f2fa-45fd-957f-a1f179b0bac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:45.422260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf8adf-f2fa-45fd-957f-a1f179b0bac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:45.422788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf8adf-f2fa-45fd-957f-a1f179b0bac6 admin admin] PATH_INFO: `/v3/users/bdeed465ce2e481fa01e6c35f44fa7f6` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:45.424732 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf8adf-f2fa-45fd-957f-a1f179b0bac6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdeed465ce2e481fa01e6c35f44fa7f6)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:45.428416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf8adf-f2fa-45fd-957f-a1f179b0bac6 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:45.452810 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1b527-06c7-47e5-8c68-22f19c6459b4 service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:45.455079 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-07d1b527-06c7-47e5-8c68-22f19c6459b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:45.474508 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1b527-06c7-47e5-8c68-22f19c6459b4 service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:45.484200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf9acd-aefa-402a-b221-1e7d4bda68c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:45.485203 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf9acd-aefa-402a-b221-1e7d4bda68c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:45.485719 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf9acd-aefa-402a-b221-1e7d4bda68c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:45.486220 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf9acd-aefa-402a-b221-1e7d4bda68c2 admin admin] PATH_INFO: `/v3/users/06df6b9b3dd742c98afae7bb81f7076e` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:45.488229 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf9acd-aefa-402a-b221-1e7d4bda68c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06df6b9b3dd742c98afae7bb81f7076e)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:45.491929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf9acd-aefa-402a-b221-1e7d4bda68c2 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:45.511561 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-bacf8adf-f2fa-45fd-957f-a1f179b0bac6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdeed465ce2e481fa01e6c35f44fa7f6'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:45.522579 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-bacf8adf-f2fa-45fd-957f-a1f179b0bac6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdeed465ce2e481fa01e6c35f44fa7f6'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:45.544247 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-bacf8adf-f2fa-45fd-957f-a1f179b0bac6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdeed465ce2e481fa01e6c35f44fa7f6'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:45.556585 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-5adf9acd-aefa-402a-b221-1e7d4bda68c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06df6b9b3dd742c98afae7bb81f7076e'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:45.557066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-bacf8adf-f2fa-45fd-957f-a1f179b0bac6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdeed465ce2e481fa01e6c35f44fa7f6'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:45.558200 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-bacf8adf-f2fa-45fd-957f-a1f179b0bac6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdeed465ce2e481fa01e6c35f44fa7f6'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:45.570380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1020/2045] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:13:45 2022] DELETE /identity/v3/users/bdeed465ce2e481fa01e6c35f44fa7f6 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:13:45.572192 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-5adf9acd-aefa-402a-b221-1e7d4bda68c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06df6b9b3dd742c98afae7bb81f7076e'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:45.579084 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e2209-9d67-421b-b073-4ed27761f4e5 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:45.579084 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e2209-9d67-421b-b073-4ed27761f4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:45.579084 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e2209-9d67-421b-b073-4ed27761f4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:45.589811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-5adf9acd-aefa-402a-b221-1e7d4bda68c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06df6b9b3dd742c98afae7bb81f7076e'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:45.599167 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-5adf9acd-aefa-402a-b221-1e7d4bda68c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06df6b9b3dd742c98afae7bb81f7076e'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:45.600026 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-5adf9acd-aefa-402a-b221-1e7d4bda68c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06df6b9b3dd742c98afae7bb81f7076e'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:45.604921 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-cd7e2209-9d67-421b-b073-4ed27761f4e5 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:45.606395 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-cd7e2209-9d67-421b-b073-4ed27761f4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:45.608651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1026/2046] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:13:45 2022] DELETE /identity/v3/users/06df6b9b3dd742c98afae7bb81f7076e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:45.613219 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96d49a-0be5-4b57-be2a-ede63e42a71d None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:45.613540 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96d49a-0be5-4b57-be2a-ede63e42a71d None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:45.613822 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96d49a-0be5-4b57-be2a-ede63e42a71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:45.639535 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-7c96d49a-0be5-4b57-be2a-ede63e42a71d None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:45.640794 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-7c96d49a-0be5-4b57-be2a-ede63e42a71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:45.710820 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1021/2047] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:13:45.726725 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e2209-9d67-421b-b073-4ed27761f4e5 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:45.730836 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-cd7e2209-9d67-421b-b073-4ed27761f4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:45.751156 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:45.768655 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1027/2048] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:45.781496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96d49a-0be5-4b57-be2a-ede63e42a71d None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:45.784381 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-7c96d49a-0be5-4b57-be2a-ede63e42a71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:45.833628 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e2209-9d67-421b-b073-4ed27761f4e5 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:45.837817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba478bc-52f6-4ad4-8d44-1e10c664c2eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:45.838436 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba478bc-52f6-4ad4-8d44-1e10c664c2eb service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:45.838714 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba478bc-52f6-4ad4-8d44-1e10c664c2eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:45.838992 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba478bc-52f6-4ad4-8d44-1e10c664c2eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:45.859341 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba478bc-52f6-4ad4-8d44-1e10c664c2eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:45.862852 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba478bc-52f6-4ad4-8d44-1e10c664c2eb service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:45.881959 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1022/2049] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:13:45.901913 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96d49a-0be5-4b57-be2a-ede63e42a71d None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:45.908598 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f5b1a-6052-43c5-9fd9-aecb514725e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:45.909675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f5b1a-6052-43c5-9fd9-aecb514725e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:45.910273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f5b1a-6052-43c5-9fd9-aecb514725e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:45.910784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f5b1a-6052-43c5-9fd9-aecb514725e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:45.928586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f5b1a-6052-43c5-9fd9-aecb514725e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:45.931320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f5b1a-6052-43c5-9fd9-aecb514725e0 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:45.952087 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1028/2050] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:13:46.359408 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba478bc-52f6-4ad4-8d44-1e10c664c2eb service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:46.362108 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-0ba478bc-52f6-4ad4-8d44-1e10c664c2eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:46.384650 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba478bc-52f6-4ad4-8d44-1e10c664c2eb service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:46.391683 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f99de-8a66-4575-802c-0cee90c7a942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:46.392364 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f99de-8a66-4575-802c-0cee90c7a942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:46.392642 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f99de-8a66-4575-802c-0cee90c7a942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:46.392907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f99de-8a66-4575-802c-0cee90c7a942 admin admin] PATH_INFO: `/v3/projects/5c4db85de834450293f4545755baae37` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:46.394093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f99de-8a66-4575-802c-0cee90c7a942 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c4db85de834450293f4545755baae37)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:46.396178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f99de-8a66-4575-802c-0cee90c7a942 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:46.410440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:46.432665 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f5b1a-6052-43c5-9fd9-aecb514725e0 service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:46.435765 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-679f5b1a-6052-43c5-9fd9-aecb514725e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:46.456866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f5b1a-6052-43c5-9fd9-aecb514725e0 service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:46.471739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-793f99de-8a66-4575-802c-0cee90c7a942 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c4db85de834450293f4545755baae37', 'request_id': 'req-793f99de-8a66-4575-802c-0cee90c7a942'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:46.479342 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-793f99de-8a66-4575-802c-0cee90c7a942 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c4db85de834450293f4545755baae37', 'request_id': 'req-793f99de-8a66-4575-802c-0cee90c7a942'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:46.489641 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-793f99de-8a66-4575-802c-0cee90c7a942 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c4db85de834450293f4545755baae37', 'request_id': 'req-793f99de-8a66-4575-802c-0cee90c7a942'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:46.505545 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-793f99de-8a66-4575-802c-0cee90c7a942 admin admin] The token cache is being invalidate because project 5c4db85de834450293f4545755baae37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:13:46.505875 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-793f99de-8a66-4575-802c-0cee90c7a942 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:46.506521 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-793f99de-8a66-4575-802c-0cee90c7a942 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:46.508104 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1023/2051] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:13:46 2022] DELETE /identity/v3/projects/5c4db85de834450293f4545755baae37 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:46.519620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49a76d-dd34-409e-a8e6-5610d04820ef None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:46.520513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49a76d-dd34-409e-a8e6-5610d04820ef None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:46.520513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49a76d-dd34-409e-a8e6-5610d04820ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:46.544460 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-5b49a76d-dd34-409e-a8e6-5610d04820ef None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:46.545888 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-5b49a76d-dd34-409e-a8e6-5610d04820ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:46.554325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569d2dd-ddef-460a-83b3-6707c550b656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:46.554972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569d2dd-ddef-460a-83b3-6707c550b656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:46.555252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569d2dd-ddef-460a-83b3-6707c550b656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:46.555524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569d2dd-ddef-460a-83b3-6707c550b656 admin admin] PATH_INFO: `/v3/projects/c24a675aa0974e878c5b49d1032af37e` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:46.556686 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8569d2dd-ddef-460a-83b3-6707c550b656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c24a675aa0974e878c5b49d1032af37e)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:46.558659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8569d2dd-ddef-460a-83b3-6707c550b656 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:46.632230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8569d2dd-ddef-460a-83b3-6707c550b656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c24a675aa0974e878c5b49d1032af37e', 'request_id': 'req-8569d2dd-ddef-460a-83b3-6707c550b656'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:46.645926 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8569d2dd-ddef-460a-83b3-6707c550b656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c24a675aa0974e878c5b49d1032af37e', 'request_id': 'req-8569d2dd-ddef-460a-83b3-6707c550b656'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:46.662396 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1024/2052] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:46.665141 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f987c-7cb4-4c52-8523-4c144f0ace7c None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:46.666294 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f987c-7cb4-4c52-8523-4c144f0ace7c None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:46.666712 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f987c-7cb4-4c52-8523-4c144f0ace7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:46.667371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8569d2dd-ddef-460a-83b3-6707c550b656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c24a675aa0974e878c5b49d1032af37e', 'request_id': 'req-8569d2dd-ddef-460a-83b3-6707c550b656'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:46.685602 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8569d2dd-ddef-460a-83b3-6707c550b656 admin admin] The token cache is being invalidate because project c24a675aa0974e878c5b49d1032af37e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:13:46.686671 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8569d2dd-ddef-460a-83b3-6707c550b656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:46.689077 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8569d2dd-ddef-460a-83b3-6707c550b656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:46.697273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1029/2053] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:13:46 2022] DELETE /identity/v3/projects/c24a675aa0974e878c5b49d1032af37e => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:46.697510 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-3a3f987c-7cb4-4c52-8523-4c144f0ace7c None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:46.698532 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-3a3f987c-7cb4-4c52-8523-4c144f0ace7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:46.699646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569d2dd-ddef-460a-83b3-6707c550b656 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:46.702802 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-8569d2dd-ddef-460a-83b3-6707c550b656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:46.772836 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:46.797131 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1025/2054] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:46.800218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f987c-7cb4-4c52-8523-4c144f0ace7c None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:46.822664 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569d2dd-ddef-460a-83b3-6707c550b656 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:46.823875 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f987c-7cb4-4c52-8523-4c144f0ace7c None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:46.841020 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc52064-691b-438d-8eb3-85aaa15539b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:46.845669 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e75a6-68ba-48f4-a657-bd40001243a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:46.847716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e75a6-68ba-48f4-a657-bd40001243a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:46.848069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e75a6-68ba-48f4-a657-bd40001243a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:46.848859 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e75a6-68ba-48f4-a657-bd40001243a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:46.849514 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc52064-691b-438d-8eb3-85aaa15539b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:46.849993 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc52064-691b-438d-8eb3-85aaa15539b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:46.850132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e75a6-68ba-48f4-a657-bd40001243a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:13:46.850437 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc52064-691b-438d-8eb3-85aaa15539b2 admin admin] PATH_INFO: `/v3/users/0c77992702bc4c28af464efb75df5aa6` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:46.850572 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e75a6-68ba-48f4-a657-bd40001243a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:46.852576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc52064-691b-438d-8eb3-85aaa15539b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c77992702bc4c28af464efb75df5aa6)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:46.853288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e75a6-68ba-48f4-a657-bd40001243a3 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:46.856139 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc52064-691b-438d-8eb3-85aaa15539b2 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:46.881680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1026/2055] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:13:46.887640 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e75a6-68ba-48f4-a657-bd40001243a3 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:46.903649 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e75a6-68ba-48f4-a657-bd40001243a3 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:46.909392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d8905-63c9-4a26-ab11-cdb30c2c4f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:46.910082 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d8905-63c9-4a26-ab11-cdb30c2c4f1d admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:46.910363 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d8905-63c9-4a26-ab11-cdb30c2c4f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:46.910634 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d8905-63c9-4a26-ab11-cdb30c2c4f1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:46.911303 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d8905-63c9-4a26-ab11-cdb30c2c4f1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:46.913314 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d8905-63c9-4a26-ab11-cdb30c2c4f1d admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:46.933323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8cc52064-691b-438d-8eb3-85aaa15539b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c77992702bc4c28af464efb75df5aa6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:46.947528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:46.953149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8cc52064-691b-438d-8eb3-85aaa15539b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c77992702bc4c28af464efb75df5aa6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:46.963851 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8cc52064-691b-438d-8eb3-85aaa15539b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c77992702bc4c28af464efb75df5aa6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:46.964107 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1027/2056] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:13:46 2022] POST /identity/v3/projects => generated 377 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:13:46.965461 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d8905-63c9-4a26-ab11-cdb30c2c4f1d admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:46.978550 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8cc52064-691b-438d-8eb3-85aaa15539b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c77992702bc4c28af464efb75df5aa6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:46.979714 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8cc52064-691b-438d-8eb3-85aaa15539b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c77992702bc4c28af464efb75df5aa6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:46.991348 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1030/2057] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:13:46 2022] DELETE /identity/v3/users/0c77992702bc4c28af464efb75df5aa6 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:13:47.081665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d8905-63c9-4a26-ab11-cdb30c2c4f1d admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:47.226772 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a5c42-1d05-4ac6-9696-7b322275fc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:47.227486 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a5c42-1d05-4ac6-9696-7b322275fc10 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:47.227936 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a5c42-1d05-4ac6-9696-7b322275fc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:47.228357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a5c42-1d05-4ac6-9696-7b322275fc10 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:47.229119 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a5c42-1d05-4ac6-9696-7b322275fc10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:47.231582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a5c42-1d05-4ac6-9696-7b322275fc10 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:47.266187 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1028/2058] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:13:46 2022] POST /identity/v3/users => generated 289 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:47.270284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc52064-691b-438d-8eb3-85aaa15539b2 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:47.271783 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-8cc52064-691b-438d-8eb3-85aaa15539b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:47.288538 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc52064-691b-438d-8eb3-85aaa15539b2 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:47.295657 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a42f63-557f-47ee-bbd9-dbc85d5747ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:47.296346 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a42f63-557f-47ee-bbd9-dbc85d5747ca admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:47.296737 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a42f63-557f-47ee-bbd9-dbc85d5747ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:47.297224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a42f63-557f-47ee-bbd9-dbc85d5747ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:47.298118 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a42f63-557f-47ee-bbd9-dbc85d5747ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:47.299760 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a42f63-557f-47ee-bbd9-dbc85d5747ca admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:47.315375 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1031/2059] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:47.318277 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a5c42-1d05-4ac6-9696-7b322275fc10 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:47.337823 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a5c42-1d05-4ac6-9696-7b322275fc10 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:47.346979 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad23d75-b64c-4d77-82d6-149657cff74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:47.348287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad23d75-b64c-4d77-82d6-149657cff74e admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:47.348674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad23d75-b64c-4d77-82d6-149657cff74e admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:47.349030 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad23d75-b64c-4d77-82d6-149657cff74e admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:47.350175 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad23d75-b64c-4d77-82d6-149657cff74e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:47.353309 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad23d75-b64c-4d77-82d6-149657cff74e admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:47.374222 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1029/2060] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:47.379954 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a42f63-557f-47ee-bbd9-dbc85d5747ca admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:47.391605 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a42f63-557f-47ee-bbd9-dbc85d5747ca admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:47.396317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c716fe-c2d9-4879-a265-4d7b82e1e8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:47.397237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c716fe-c2d9-4879-a265-4d7b82e1e8c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:47.397526 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c716fe-c2d9-4879-a265-4d7b82e1e8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:47.397831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c716fe-c2d9-4879-a265-4d7b82e1e8c9 admin admin] PATH_INFO: `/v3/projects/79a173ba61d54c9eaf06e0c960ac1249/users/309ca8171b0345f59697148cc62c2409/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:47.421884 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c716fe-c2d9-4879-a265-4d7b82e1e8c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79a173ba61d54c9eaf06e0c960ac1249, user_id=309ca8171b0345f59697148cc62c2409, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:47.424845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c716fe-c2d9-4879-a265-4d7b82e1e8c9 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:47.442910 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1032/2061] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:47 2022] PUT /identity/v3/projects/79a173ba61d54c9eaf06e0c960ac1249/users/309ca8171b0345f59697148cc62c2409/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:13:47.451195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad23d75-b64c-4d77-82d6-149657cff74e admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:47.455180 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-2ad23d75-b64c-4d77-82d6-149657cff74e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:47.461897 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a98f01-9421-4ea0-b69a-80ad9152a766 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:47.462375 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a98f01-9421-4ea0-b69a-80ad9152a766 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:47.462852 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a98f01-9421-4ea0-b69a-80ad9152a766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:47.480385 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad23d75-b64c-4d77-82d6-149657cff74e admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:47.514733 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-89a98f01-9421-4ea0-b69a-80ad9152a766 None None] MFA Rules not processed for user `309ca8171b0345f59697148cc62c2409`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:47.516298 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-89a98f01-9421-4ea0-b69a-80ad9152a766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:47.559454 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d29b3-e4ef-47e1-981d-58990dfb2b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:47.560836 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d29b3-e4ef-47e1-981d-58990dfb2b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:47.561243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d29b3-e4ef-47e1-981d-58990dfb2b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:47.561690 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d29b3-e4ef-47e1-981d-58990dfb2b61 admin admin] PATH_INFO: `/v3/projects/aad2e0d8f0da4a6b8feade51237cecf2` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:47.563409 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d29b3-e4ef-47e1-981d-58990dfb2b61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aad2e0d8f0da4a6b8feade51237cecf2)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:47.566799 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d29b3-e4ef-47e1-981d-58990dfb2b61 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:47.608802 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1033/2062] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:13:47.612971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:47.620578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a98f01-9421-4ea0-b69a-80ad9152a766 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:47.622226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-89a98f01-9421-4ea0-b69a-80ad9152a766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:47.628849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:47.642802 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a25d29b3-e4ef-47e1-981d-58990dfb2b61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aad2e0d8f0da4a6b8feade51237cecf2', 'request_id': 'req-a25d29b3-e4ef-47e1-981d-58990dfb2b61'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:47.652506 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a25d29b3-e4ef-47e1-981d-58990dfb2b61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aad2e0d8f0da4a6b8feade51237cecf2', 'request_id': 'req-a25d29b3-e4ef-47e1-981d-58990dfb2b61'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:47.662084 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a25d29b3-e4ef-47e1-981d-58990dfb2b61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aad2e0d8f0da4a6b8feade51237cecf2', 'request_id': 'req-a25d29b3-e4ef-47e1-981d-58990dfb2b61'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:47.679599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a25d29b3-e4ef-47e1-981d-58990dfb2b61 admin admin] The token cache is being invalidate because project aad2e0d8f0da4a6b8feade51237cecf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:13:47.679960 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a25d29b3-e4ef-47e1-981d-58990dfb2b61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:47.680577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a25d29b3-e4ef-47e1-981d-58990dfb2b61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:47.682605 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1030/2063] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:13:47 2022] DELETE /identity/v3/projects/aad2e0d8f0da4a6b8feade51237cecf2 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:13:47.687263 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1dcdf-905b-476b-b46c-8d5cbbeb1ec3 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:47.687594 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1dcdf-905b-476b-b46c-8d5cbbeb1ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:47.687853 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1dcdf-905b-476b-b46c-8d5cbbeb1ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:47.710330 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-edd1dcdf-905b-476b-b46c-8d5cbbeb1ec3 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:47.711569 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-edd1dcdf-905b-476b-b46c-8d5cbbeb1ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:47.714771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a98f01-9421-4ea0-b69a-80ad9152a766 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:47.794231 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19246ae6-320d-4c9c-87d7-6917c0747471 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:47.795614 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19246ae6-320d-4c9c-87d7-6917c0747471 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:47.795895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19246ae6-320d-4c9c-87d7-6917c0747471 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:47.796328 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19246ae6-320d-4c9c-87d7-6917c0747471 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:47.798567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-19246ae6-320d-4c9c-87d7-6917c0747471 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:47.802728 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1031/2064] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:13:47.818458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e17f7-3c49-47a1-8f35-ee20d86f9e04 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:47.818943 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e17f7-3c49-47a1-8f35-ee20d86f9e04 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:47.819422 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e17f7-3c49-47a1-8f35-ee20d86f9e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:47.851784 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-fa6e17f7-3c49-47a1-8f35-ee20d86f9e04 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:47.853198 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-fa6e17f7-3c49-47a1-8f35-ee20d86f9e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:47.874792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1032/2065] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:13:47.878282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e17f7-3c49-47a1-8f35-ee20d86f9e04 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:47.898421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e17f7-3c49-47a1-8f35-ee20d86f9e04 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:47.908714 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b6873-2418-4549-902d-010796cc6bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:47.909846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b6873-2418-4549-902d-010796cc6bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:47.910735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b6873-2418-4549-902d-010796cc6bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:47.911213 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b6873-2418-4549-902d-010796cc6bb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:47.912485 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b6873-2418-4549-902d-010796cc6bb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:13:47.912991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b6873-2418-4549-902d-010796cc6bb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:47.916696 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b6873-2418-4549-902d-010796cc6bb4 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:47.917782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19246ae6-320d-4c9c-87d7-6917c0747471 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:47.922985 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19246ae6-320d-4c9c-87d7-6917c0747471 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:47.942972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1034/2066] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:13:47.957377 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19246ae6-320d-4c9c-87d7-6917c0747471 service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:47.960917 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1033/2067] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:13:47.984496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19246ae6-320d-4c9c-87d7-6917c0747471 service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:47.998796 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa60605-61b9-4dbf-97fd-113aece10fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:47.999988 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa60605-61b9-4dbf-97fd-113aece10fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:48.000548 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa60605-61b9-4dbf-97fd-113aece10fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:48.001040 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa60605-61b9-4dbf-97fd-113aece10fd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:48.002348 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa60605-61b9-4dbf-97fd-113aece10fd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:48.005948 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa60605-61b9-4dbf-97fd-113aece10fd9 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:48.072672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1035/2068] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:13:47 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:48.078452 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b6873-2418-4549-902d-010796cc6bb4 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:48.097271 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b6873-2418-4549-902d-010796cc6bb4 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:48.201790 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf5531-2e76-4b4b-be30-fe9e8cd49b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:48.202942 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf5531-2e76-4b4b-be30-fe9e8cd49b95 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:48.203492 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf5531-2e76-4b4b-be30-fe9e8cd49b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:48.203942 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf5531-2e76-4b4b-be30-fe9e8cd49b95 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:48.205026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf5531-2e76-4b4b-be30-fe9e8cd49b95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:48.208679 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf5531-2e76-4b4b-be30-fe9e8cd49b95 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:48.263122 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa60605-61b9-4dbf-97fd-113aece10fd9 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:48.263877 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1034/2069] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:13:48 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:48.283021 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa60605-61b9-4dbf-97fd-113aece10fd9 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:48.294585 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323baa0-6ff3-473a-80cc-8cbe131ece50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:48.296529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323baa0-6ff3-473a-80cc-8cbe131ece50 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:48.297093 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323baa0-6ff3-473a-80cc-8cbe131ece50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:48.297619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323baa0-6ff3-473a-80cc-8cbe131ece50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:48.298930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b323baa0-6ff3-473a-80cc-8cbe131ece50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:48.301786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b323baa0-6ff3-473a-80cc-8cbe131ece50 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:48.321888 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1036/2070] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:48.328100 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf5531-2e76-4b4b-be30-fe9e8cd49b95 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:48.344484 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf5531-2e76-4b4b-be30-fe9e8cd49b95 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:48.352587 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ccaeb-cd64-4d06-aa1f-447cfdc81442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:48.353978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ccaeb-cd64-4d06-aa1f-447cfdc81442 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:48.354496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ccaeb-cd64-4d06-aa1f-447cfdc81442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:48.354979 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ccaeb-cd64-4d06-aa1f-447cfdc81442 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:48.356164 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ccaeb-cd64-4d06-aa1f-447cfdc81442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:48.360472 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ccaeb-cd64-4d06-aa1f-447cfdc81442 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:48.376596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1035/2071] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:48.382036 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323baa0-6ff3-473a-80cc-8cbe131ece50 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:48.408418 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323baa0-6ff3-473a-80cc-8cbe131ece50 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:48.424003 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae2ce6-be70-42d5-a917-b9a5da8d4c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:48.424761 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae2ce6-be70-42d5-a917-b9a5da8d4c51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:48.425142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae2ce6-be70-42d5-a917-b9a5da8d4c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:48.425418 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae2ce6-be70-42d5-a917-b9a5da8d4c51 admin admin] PATH_INFO: `/v3/projects/20371057dd0e41c6988ab054b705b886/users/b75d3725088d4796826da25b53a85180/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:48.471500 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae2ce6-be70-42d5-a917-b9a5da8d4c51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20371057dd0e41c6988ab054b705b886, user_id=b75d3725088d4796826da25b53a85180, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:48.477421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae2ce6-be70-42d5-a917-b9a5da8d4c51 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:48.510782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1037/2072] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:48 2022] PUT /identity/v3/projects/20371057dd0e41c6988ab054b705b886/users/b75d3725088d4796826da25b53a85180/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:13:48.530409 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dddbf7-fc3e-470c-9fc4-76c3ef9dd78f None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:48.530865 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dddbf7-fc3e-470c-9fc4-76c3ef9dd78f None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:48.530981 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dddbf7-fc3e-470c-9fc4-76c3ef9dd78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:48.571406 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-77dddbf7-fc3e-470c-9fc4-76c3ef9dd78f None None] MFA Rules not processed for user `b75d3725088d4796826da25b53a85180`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:48.572706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-77dddbf7-fc3e-470c-9fc4-76c3ef9dd78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:48.577634 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:48.681271 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1036/2073] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:13:48.687792 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae2ce6-be70-42d5-a917-b9a5da8d4c51 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:48.719744 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:48.782285 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae2ce6-be70-42d5-a917-b9a5da8d4c51 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:48.787332 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83624146-6704-44e3-a804-d14d85d2a2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:48.787979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83624146-6704-44e3-a804-d14d85d2a2b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:48.788255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83624146-6704-44e3-a804-d14d85d2a2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:48.788525 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83624146-6704-44e3-a804-d14d85d2a2b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:48.789181 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83624146-6704-44e3-a804-d14d85d2a2b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:48.791538 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83624146-6704-44e3-a804-d14d85d2a2b9 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:48.826670 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1038/2074] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:13:48 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:48.830975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dddbf7-fc3e-470c-9fc4-76c3ef9dd78f None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:48.843809 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dddbf7-fc3e-470c-9fc4-76c3ef9dd78f None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:48.913128 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681805c1-d9db-4f27-8f3b-05a48a06d11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:48.913776 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681805c1-d9db-4f27-8f3b-05a48a06d11f admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:48.914083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681805c1-d9db-4f27-8f3b-05a48a06d11f admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:48.914352 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681805c1-d9db-4f27-8f3b-05a48a06d11f admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:48.914988 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681805c1-d9db-4f27-8f3b-05a48a06d11f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:48.916930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681805c1-d9db-4f27-8f3b-05a48a06d11f admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:48.944580 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1037/2075] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:13:48 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:48.949138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83624146-6704-44e3-a804-d14d85d2a2b9 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:48.961355 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83624146-6704-44e3-a804-d14d85d2a2b9 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:48.967355 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e068fd-fa37-4ed4-ac61-455fa4a7658f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:48.968132 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e068fd-fa37-4ed4-ac61-455fa4a7658f admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:48.968451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e068fd-fa37-4ed4-ac61-455fa4a7658f admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:48.968785 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e068fd-fa37-4ed4-ac61-455fa4a7658f admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:48.969488 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e068fd-fa37-4ed4-ac61-455fa4a7658f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:48.971312 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e068fd-fa37-4ed4-ac61-455fa4a7658f admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:48.982977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1039/2076] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:13:48.988284 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681805c1-d9db-4f27-8f3b-05a48a06d11f admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:49.005915 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681805c1-d9db-4f27-8f3b-05a48a06d11f admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:49.012382 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada764ef-943f-407e-9863-aa217434299d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:49.013060 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada764ef-943f-407e-9863-aa217434299d admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:49.013947 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada764ef-943f-407e-9863-aa217434299d admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:49.014263 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada764ef-943f-407e-9863-aa217434299d admin admin] PATH_INFO: `/v3/projects/1749f04b1a4a4b66a3ae50fd5be8f9d3/users/7dadc57e7dc1410889a5692b36800895/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:49.033992 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada764ef-943f-407e-9863-aa217434299d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1749f04b1a4a4b66a3ae50fd5be8f9d3, user_id=7dadc57e7dc1410889a5692b36800895, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:49.039458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada764ef-943f-407e-9863-aa217434299d admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:49.064098 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1038/2077] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:48 2022] PUT /identity/v3/projects/1749f04b1a4a4b66a3ae50fd5be8f9d3/users/7dadc57e7dc1410889a5692b36800895/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:49.069866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e068fd-fa37-4ed4-ac61-455fa4a7658f admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:49.081185 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e068fd-fa37-4ed4-ac61-455fa4a7658f admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:49.161464 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f29db9-7e76-43ea-9b55-ee7172564777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:49.162243 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f29db9-7e76-43ea-9b55-ee7172564777 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:49.162520 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f29db9-7e76-43ea-9b55-ee7172564777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:49.162790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f29db9-7e76-43ea-9b55-ee7172564777 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:49.163470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f29db9-7e76-43ea-9b55-ee7172564777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:49.165892 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f29db9-7e76-43ea-9b55-ee7172564777 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:49.180326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1040/2078] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:13:49.187218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada764ef-943f-407e-9863-aa217434299d admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:49.203371 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada764ef-943f-407e-9863-aa217434299d admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:49.210993 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0b42f-dc26-47da-a7d1-8e6bba28625d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:49.212143 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0b42f-dc26-47da-a7d1-8e6bba28625d admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:49.212601 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0b42f-dc26-47da-a7d1-8e6bba28625d admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:49.213046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0b42f-dc26-47da-a7d1-8e6bba28625d admin admin] PATH_INFO: `/v3/projects/1749f04b1a4a4b66a3ae50fd5be8f9d3/users/7dadc57e7dc1410889a5692b36800895/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:49.216176 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0b42f-dc26-47da-a7d1-8e6bba28625d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1749f04b1a4a4b66a3ae50fd5be8f9d3, user_id=7dadc57e7dc1410889a5692b36800895, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:49.221402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0b42f-dc26-47da-a7d1-8e6bba28625d admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:49.239624 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1039/2079] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:49 2022] PUT /identity/v3/projects/1749f04b1a4a4b66a3ae50fd5be8f9d3/users/7dadc57e7dc1410889a5692b36800895/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:49.256879 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94495c9b-d938-4bc4-b05f-65b6b62c489b None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:49.257231 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94495c9b-d938-4bc4-b05f-65b6b62c489b None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:49.257543 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94495c9b-d938-4bc4-b05f-65b6b62c489b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:49.303864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-94495c9b-d938-4bc4-b05f-65b6b62c489b None None] MFA Rules not processed for user `7dadc57e7dc1410889a5692b36800895`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:49.305217 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-94495c9b-d938-4bc4-b05f-65b6b62c489b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:49.453845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1041/2080] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:13:49.492758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0b42f-dc26-47da-a7d1-8e6bba28625d admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:49.496119 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-fbf0b42f-dc26-47da-a7d1-8e6bba28625d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:49.581561 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0b42f-dc26-47da-a7d1-8e6bba28625d admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:49.588630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca6651-1436-43d7-a5a4-258a68571729 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:49.589249 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca6651-1436-43d7-a5a4-258a68571729 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:49.589525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca6651-1436-43d7-a5a4-258a68571729 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:49.589867 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca6651-1436-43d7-a5a4-258a68571729 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:49.601466 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:49.667398 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca6651-1436-43d7-a5a4-258a68571729 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:49.669059 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca6651-1436-43d7-a5a4-258a68571729 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:49.681134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1040/2081] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:13:51.434365 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94495c9b-d938-4bc4-b05f-65b6b62c489b None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:51.444255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94495c9b-d938-4bc4-b05f-65b6b62c489b None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:51.448517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199783f5-aeef-4fec-bc58-219b7ca5244c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:51.449388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199783f5-aeef-4fec-bc58-219b7ca5244c service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:51.449834 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199783f5-aeef-4fec-bc58-219b7ca5244c service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:51.450130 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199783f5-aeef-4fec-bc58-219b7ca5244c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:51.452240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-199783f5-aeef-4fec-bc58-219b7ca5244c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:51.467481 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:51.550145 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199783f5-aeef-4fec-bc58-219b7ca5244c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:51.553018 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199783f5-aeef-4fec-bc58-219b7ca5244c service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:51.569189 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1042/2082] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:13:56.173326 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca6651-1436-43d7-a5a4-258a68571729 service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:56.176732 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-e1ca6651-1436-43d7-a5a4-258a68571729 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:56.348165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca6651-1436-43d7-a5a4-258a68571729 service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:56.358929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f6d50-f1f4-4907-ad1a-618a58e70271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:56.360223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f6d50-f1f4-4907-ad1a-618a58e70271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:56.360907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f6d50-f1f4-4907-ad1a-618a58e70271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:56.361348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f6d50-f1f4-4907-ad1a-618a58e70271 admin admin] PATH_INFO: `/v3/users/df57699e3cd1494d981ddb451a56c899` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:56.365572 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f6d50-f1f4-4907-ad1a-618a58e70271 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df57699e3cd1494d981ddb451a56c899)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:56.367591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f6d50-f1f4-4907-ad1a-618a58e70271 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:56.468847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-709f6d50-f1f4-4907-ad1a-618a58e70271 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df57699e3cd1494d981ddb451a56c899'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:56.489293 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-709f6d50-f1f4-4907-ad1a-618a58e70271 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df57699e3cd1494d981ddb451a56c899'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:56.508949 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-709f6d50-f1f4-4907-ad1a-618a58e70271 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df57699e3cd1494d981ddb451a56c899'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:56.520946 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-709f6d50-f1f4-4907-ad1a-618a58e70271 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df57699e3cd1494d981ddb451a56c899'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:56.521814 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-709f6d50-f1f4-4907-ad1a-618a58e70271 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df57699e3cd1494d981ddb451a56c899'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:56.533946 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1041/2083] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:13:56 2022] DELETE /identity/v3/users/df57699e3cd1494d981ddb451a56c899 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:56.536810 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2a560-bbb8-4ee5-ac32-50e3643903fd None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:56.537336 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2a560-bbb8-4ee5-ac32-50e3643903fd None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:56.537336 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2a560-bbb8-4ee5-ac32-50e3643903fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:56.568968 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-86b2a560-bbb8-4ee5-ac32-50e3643903fd None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:56.570486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-86b2a560-bbb8-4ee5-ac32-50e3643903fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:56.696153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1043/2084] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:56.707382 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f6d50-f1f4-4907-ad1a-618a58e70271 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:56.708916 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-709f6d50-f1f4-4907-ad1a-618a58e70271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:56.749154 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:56.782150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f6d50-f1f4-4907-ad1a-618a58e70271 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:56.786072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5aa65-5c32-49d8-bc68-9e4a4c1bcaf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:56.786703 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5aa65-5c32-49d8-bc68-9e4a4c1bcaf0 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:56.786985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5aa65-5c32-49d8-bc68-9e4a4c1bcaf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:56.787259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5aa65-5c32-49d8-bc68-9e4a4c1bcaf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:56.797698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b5aa65-5c32-49d8-bc68-9e4a4c1bcaf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:56.799187 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b5aa65-5c32-49d8-bc68-9e4a4c1bcaf0 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:56.810005 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1042/2085] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:13:57.146168 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2a560-bbb8-4ee5-ac32-50e3643903fd None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:57.147990 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-86b2a560-bbb8-4ee5-ac32-50e3643903fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:57.162272 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2a560-bbb8-4ee5-ac32-50e3643903fd None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:57.168311 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae5823-1c0b-4c89-af4d-5be6c5c3133d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:57.168987 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae5823-1c0b-4c89-af4d-5be6c5c3133d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:57.169274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae5823-1c0b-4c89-af4d-5be6c5c3133d admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:57.170522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae5823-1c0b-4c89-af4d-5be6c5c3133d admin admin] PATH_INFO: `/v3/projects/67fb63da3ed9417b9fc7476a4b822e6a` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:57.171944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae5823-1c0b-4c89-af4d-5be6c5c3133d admin admin] RBAC: Authorizing `identity:delete_project(project_id=67fb63da3ed9417b9fc7476a4b822e6a)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:57.174015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae5823-1c0b-4c89-af4d-5be6c5c3133d admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:57.250856 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1aae5823-1c0b-4c89-af4d-5be6c5c3133d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67fb63da3ed9417b9fc7476a4b822e6a', 'request_id': 'req-1aae5823-1c0b-4c89-af4d-5be6c5c3133d'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:57.260834 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1aae5823-1c0b-4c89-af4d-5be6c5c3133d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67fb63da3ed9417b9fc7476a4b822e6a', 'request_id': 'req-1aae5823-1c0b-4c89-af4d-5be6c5c3133d'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:57.279502 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1aae5823-1c0b-4c89-af4d-5be6c5c3133d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67fb63da3ed9417b9fc7476a4b822e6a', 'request_id': 'req-1aae5823-1c0b-4c89-af4d-5be6c5c3133d'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:57.296336 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1aae5823-1c0b-4c89-af4d-5be6c5c3133d admin admin] The token cache is being invalidate because project 67fb63da3ed9417b9fc7476a4b822e6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:13:57.296611 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1aae5823-1c0b-4c89-af4d-5be6c5c3133d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:57.297393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1aae5823-1c0b-4c89-af4d-5be6c5c3133d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:13:57.299044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1044/2086] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:13:57 2022] DELETE /identity/v3/projects/67fb63da3ed9417b9fc7476a4b822e6a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:57.306055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67657146-a512-476f-a704-5aaaf83f628e None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:57.306905 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67657146-a512-476f-a704-5aaaf83f628e None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:57.307407 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67657146-a512-476f-a704-5aaaf83f628e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:57.330373 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-67657146-a512-476f-a704-5aaaf83f628e None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:57.331625 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-67657146-a512-476f-a704-5aaaf83f628e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:57.412776 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1043/2087] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:57.427767 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481c9d8-c1da-491c-a017-fd6a039ddec8 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:57.428235 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481c9d8-c1da-491c-a017-fd6a039ddec8 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:57.428721 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481c9d8-c1da-491c-a017-fd6a039ddec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:57.459159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-3481c9d8-c1da-491c-a017-fd6a039ddec8 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:57.460872 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-3481c9d8-c1da-491c-a017-fd6a039ddec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:57.469986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1045/2088] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:57.474490 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67657146-a512-476f-a704-5aaaf83f628e None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:57.487221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67657146-a512-476f-a704-5aaaf83f628e None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:57.492935 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f81fb9-1381-49e4-b865-d5b47ca8c0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:57.493625 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f81fb9-1381-49e4-b865-d5b47ca8c0ee admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:57.493935 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f81fb9-1381-49e4-b865-d5b47ca8c0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:57.494211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f81fb9-1381-49e4-b865-d5b47ca8c0ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:57.494880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f81fb9-1381-49e4-b865-d5b47ca8c0ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:13:57.495375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f81fb9-1381-49e4-b865-d5b47ca8c0ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:57.497027 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f81fb9-1381-49e4-b865-d5b47ca8c0ee admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:57.511426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1044/2089] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:13:57.517605 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481c9d8-c1da-491c-a017-fd6a039ddec8 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:57.534339 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481c9d8-c1da-491c-a017-fd6a039ddec8 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:57.541641 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae190abd-9d6d-4215-81a0-b7a5baf267c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:57.542513 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae190abd-9d6d-4215-81a0-b7a5baf267c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:57.542984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae190abd-9d6d-4215-81a0-b7a5baf267c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:57.543435 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae190abd-9d6d-4215-81a0-b7a5baf267c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:57.544477 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae190abd-9d6d-4215-81a0-b7a5baf267c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:57.548126 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae190abd-9d6d-4215-81a0-b7a5baf267c0 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:57.577652 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:57.590067 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1046/2090] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:13:57 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:57.593606 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f81fb9-1381-49e4-b865-d5b47ca8c0ee admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:57.606014 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f81fb9-1381-49e4-b865-d5b47ca8c0ee admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:57.702699 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1851b-f36a-4659-b312-769794e7b8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:57.703685 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1851b-f36a-4659-b312-769794e7b8a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:57.704227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1851b-f36a-4659-b312-769794e7b8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:57.704697 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1851b-f36a-4659-b312-769794e7b8a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:57.705886 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e1851b-f36a-4659-b312-769794e7b8a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:57.709524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e1851b-f36a-4659-b312-769794e7b8a5 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:57.756746 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1045/2091] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:13:57 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:57.761959 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae190abd-9d6d-4215-81a0-b7a5baf267c0 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:57.782145 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae190abd-9d6d-4215-81a0-b7a5baf267c0 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:57.790400 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43325f14-7211-4153-8c43-3834cbc3dbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:57.791469 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43325f14-7211-4153-8c43-3834cbc3dbf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:57.791983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43325f14-7211-4153-8c43-3834cbc3dbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:57.792382 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43325f14-7211-4153-8c43-3834cbc3dbf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:57.793171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43325f14-7211-4153-8c43-3834cbc3dbf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:57.795568 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43325f14-7211-4153-8c43-3834cbc3dbf4 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:57.813076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1047/2092] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:57.817351 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1851b-f36a-4659-b312-769794e7b8a5 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:57.831416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1851b-f36a-4659-b312-769794e7b8a5 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:57.839829 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a31de3-ee29-4dc6-84d4-6848facbfa8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:57.840586 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a31de3-ee29-4dc6-84d4-6848facbfa8b admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:57.841101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a31de3-ee29-4dc6-84d4-6848facbfa8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:57.841475 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a31de3-ee29-4dc6-84d4-6848facbfa8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:57.843066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a31de3-ee29-4dc6-84d4-6848facbfa8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:57.845523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a31de3-ee29-4dc6-84d4-6848facbfa8b admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:57.859407 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1046/2093] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:57.864310 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43325f14-7211-4153-8c43-3834cbc3dbf4 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:57.878441 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43325f14-7211-4153-8c43-3834cbc3dbf4 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:57.883596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820c8cf-ff34-421e-a3ae-846cb15fde01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:57.884266 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820c8cf-ff34-421e-a3ae-846cb15fde01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:57.884540 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820c8cf-ff34-421e-a3ae-846cb15fde01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:57.884813 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820c8cf-ff34-421e-a3ae-846cb15fde01 admin admin] PATH_INFO: `/v3/projects/88723982e96f47c68bb1ad26a49ddece/users/c2e1753f05db4a1b8a2dd00041aa09a7/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:57.903404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820c8cf-ff34-421e-a3ae-846cb15fde01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88723982e96f47c68bb1ad26a49ddece, user_id=c2e1753f05db4a1b8a2dd00041aa09a7, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:57.907198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820c8cf-ff34-421e-a3ae-846cb15fde01 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:57.922404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1048/2094] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:57 2022] PUT /identity/v3/projects/88723982e96f47c68bb1ad26a49ddece/users/c2e1753f05db4a1b8a2dd00041aa09a7/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:57.933375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8696d98-f387-48f0-8498-9d20ef20ec58 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:57.933653 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8696d98-f387-48f0-8498-9d20ef20ec58 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:57.933976 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8696d98-f387-48f0-8498-9d20ef20ec58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:57.967358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-a8696d98-f387-48f0-8498-9d20ef20ec58 None None] MFA Rules not processed for user `c2e1753f05db4a1b8a2dd00041aa09a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:57.968793 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-a8696d98-f387-48f0-8498-9d20ef20ec58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:58.006318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:58.051881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1047/2095] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:58.056790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820c8cf-ff34-421e-a3ae-846cb15fde01 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:58.155329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820c8cf-ff34-421e-a3ae-846cb15fde01 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:58.160717 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dbdab-857a-4eef-a2e9-50defabbb2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:58.161477 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dbdab-857a-4eef-a2e9-50defabbb2da admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:58.161904 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dbdab-857a-4eef-a2e9-50defabbb2da admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:58.162281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dbdab-857a-4eef-a2e9-50defabbb2da admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:58.163040 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337dbdab-857a-4eef-a2e9-50defabbb2da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:58.165308 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337dbdab-857a-4eef-a2e9-50defabbb2da admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:58.197398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1049/2096] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:13:58 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:58.203031 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8696d98-f387-48f0-8498-9d20ef20ec58 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:58.221025 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8696d98-f387-48f0-8498-9d20ef20ec58 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:58.307118 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae8396-7b19-48db-a0a2-06a7d16b100a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:58.307859 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae8396-7b19-48db-a0a2-06a7d16b100a admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:58.308149 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae8396-7b19-48db-a0a2-06a7d16b100a admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:58.308427 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae8396-7b19-48db-a0a2-06a7d16b100a admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:58.309087 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae8396-7b19-48db-a0a2-06a7d16b100a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:58.311953 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae8396-7b19-48db-a0a2-06a7d16b100a admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:58.340546 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1048/2097] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:13:58 2022] POST /identity/v3/users => generated 292 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:58.347352 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dbdab-857a-4eef-a2e9-50defabbb2da admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:58.362067 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dbdab-857a-4eef-a2e9-50defabbb2da admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:58.367136 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be5f70-3580-44b8-bc6e-0e3fdb958d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:58.367772 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be5f70-3580-44b8-bc6e-0e3fdb958d81 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:58.368246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be5f70-3580-44b8-bc6e-0e3fdb958d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:58.368525 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be5f70-3580-44b8-bc6e-0e3fdb958d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:58.369190 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be5f70-3580-44b8-bc6e-0e3fdb958d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:58.371122 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be5f70-3580-44b8-bc6e-0e3fdb958d81 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:58.383724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1050/2098] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:58.387911 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae8396-7b19-48db-a0a2-06a7d16b100a admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:58.399063 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae8396-7b19-48db-a0a2-06a7d16b100a admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:58.404159 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1921f9-ac7c-461a-b729-0405a716955f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:58.404942 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1921f9-ac7c-461a-b729-0405a716955f admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:58.405228 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1921f9-ac7c-461a-b729-0405a716955f admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:58.405592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1921f9-ac7c-461a-b729-0405a716955f admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:58.406303 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1921f9-ac7c-461a-b729-0405a716955f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:58.408030 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1921f9-ac7c-461a-b729-0405a716955f admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:58.426285 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1049/2099] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:58.432601 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be5f70-3580-44b8-bc6e-0e3fdb958d81 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:58.447746 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be5f70-3580-44b8-bc6e-0e3fdb958d81 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:58.455097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26de52a-485b-4960-a422-33d7d75882c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:58.455776 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26de52a-485b-4960-a422-33d7d75882c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:58.456067 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26de52a-485b-4960-a422-33d7d75882c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:58.456340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26de52a-485b-4960-a422-33d7d75882c4 admin admin] PATH_INFO: `/v3/projects/61d68ea03ca24388b663ed895d5fe9d8/users/6ed0078f192045bab2573723c3c666d2/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:58.486634 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26de52a-485b-4960-a422-33d7d75882c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61d68ea03ca24388b663ed895d5fe9d8, user_id=6ed0078f192045bab2573723c3c666d2, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:58.490145 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26de52a-485b-4960-a422-33d7d75882c4 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:58.515289 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1051/2100] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:58 2022] PUT /identity/v3/projects/61d68ea03ca24388b663ed895d5fe9d8/users/6ed0078f192045bab2573723c3c666d2/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:58.533497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfd7c2-28db-4a6a-a59a-4c077e1d7d7b None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:58.533841 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfd7c2-28db-4a6a-a59a-4c077e1d7d7b None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:58.534091 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfd7c2-28db-4a6a-a59a-4c077e1d7d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:58.573817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-6ddfd7c2-28db-4a6a-a59a-4c077e1d7d7b None None] MFA Rules not processed for user `6ed0078f192045bab2573723c3c666d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:58.576616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-6ddfd7c2-28db-4a6a-a59a-4c077e1d7d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:58.688617 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1050/2101] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:13:58.693315 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26de52a-485b-4960-a422-33d7d75882c4 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:58.762660 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26de52a-485b-4960-a422-33d7d75882c4 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:58.768160 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd5637-9901-4206-987c-1f17294c6689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:58.768841 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd5637-9901-4206-987c-1f17294c6689 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:58.769172 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd5637-9901-4206-987c-1f17294c6689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:58.769493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd5637-9901-4206-987c-1f17294c6689 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:58.770257 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd5637-9901-4206-987c-1f17294c6689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:58.772359 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd5637-9901-4206-987c-1f17294c6689 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:58.803230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:58.817631 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1052/2102] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:13:58 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:58.823968 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfd7c2-28db-4a6a-a59a-4c077e1d7d7b None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:58.843973 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfd7c2-28db-4a6a-a59a-4c077e1d7d7b None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:58.896277 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:13:58.926138 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e5497-aff3-4c7d-9bc8-9d197ff44c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:58.926750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e5497-aff3-4c7d-9bc8-9d197ff44c11 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:58.927023 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e5497-aff3-4c7d-9bc8-9d197ff44c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:58.927295 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e5497-aff3-4c7d-9bc8-9d197ff44c11 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:58.927925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e5497-aff3-4c7d-9bc8-9d197ff44c11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:58.929879 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e5497-aff3-4c7d-9bc8-9d197ff44c11 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:58.955902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1051/2103] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:13:58 2022] POST /identity/v3/users => generated 299 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:13:58.961314 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd5637-9901-4206-987c-1f17294c6689 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:58.977397 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd5637-9901-4206-987c-1f17294c6689 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:58.986010 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00072dd4-8723-48b2-9896-76f1c22fa0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:58.986918 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00072dd4-8723-48b2-9896-76f1c22fa0f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:58.987355 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00072dd4-8723-48b2-9896-76f1c22fa0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:58.987727 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00072dd4-8723-48b2-9896-76f1c22fa0f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:58.988551 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00072dd4-8723-48b2-9896-76f1c22fa0f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:58.991123 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00072dd4-8723-48b2-9896-76f1c22fa0f4 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:59.003847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1053/2104] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:59.009271 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e5497-aff3-4c7d-9bc8-9d197ff44c11 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:59.026692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e5497-aff3-4c7d-9bc8-9d197ff44c11 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:59.035403 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b42ed6-7ec6-4eee-8e23-55f4158286b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:59.036474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b42ed6-7ec6-4eee-8e23-55f4158286b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:59.036925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b42ed6-7ec6-4eee-8e23-55f4158286b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:59.037367 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b42ed6-7ec6-4eee-8e23-55f4158286b8 admin admin] PATH_INFO: `/v3/projects/69f3b458bf5549aba6d9a92cf937157d/users/eb4c352e955c4b60a580428ff5d8f824/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:59.064639 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b42ed6-7ec6-4eee-8e23-55f4158286b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69f3b458bf5549aba6d9a92cf937157d, user_id=eb4c352e955c4b60a580428ff5d8f824, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:59.068744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b42ed6-7ec6-4eee-8e23-55f4158286b8 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:59.095170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1052/2105] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:59 2022] PUT /identity/v3/projects/69f3b458bf5549aba6d9a92cf937157d/users/eb4c352e955c4b60a580428ff5d8f824/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:13:59.099821 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00072dd4-8723-48b2-9896-76f1c22fa0f4 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:59.116401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00072dd4-8723-48b2-9896-76f1c22fa0f4 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:59.201553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5a298-fd6d-4553-99fa-e610c20136c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:59.202538 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5a298-fd6d-4553-99fa-e610c20136c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:59.203834 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5a298-fd6d-4553-99fa-e610c20136c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:59.204282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5a298-fd6d-4553-99fa-e610c20136c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:59.205364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5a298-fd6d-4553-99fa-e610c20136c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:59.208190 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5a298-fd6d-4553-99fa-e610c20136c1 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:59.226639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1054/2106] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:13:59.234295 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b42ed6-7ec6-4eee-8e23-55f4158286b8 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:59.250422 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b42ed6-7ec6-4eee-8e23-55f4158286b8 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:59.258516 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f1b12-fc26-43e6-9859-a6d43c5b3548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:59.259625 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f1b12-fc26-43e6-9859-a6d43c5b3548 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:59.260105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f1b12-fc26-43e6-9859-a6d43c5b3548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:59.260578 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f1b12-fc26-43e6-9859-a6d43c5b3548 admin admin] PATH_INFO: `/v3/projects/69f3b458bf5549aba6d9a92cf937157d/users/eb4c352e955c4b60a580428ff5d8f824/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:59.263579 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f1b12-fc26-43e6-9859-a6d43c5b3548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69f3b458bf5549aba6d9a92cf937157d, user_id=eb4c352e955c4b60a580428ff5d8f824, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:59.266364 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f1b12-fc26-43e6-9859-a6d43c5b3548 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:59.278498 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1053/2107] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:13:59 2022] PUT /identity/v3/projects/69f3b458bf5549aba6d9a92cf937157d/users/eb4c352e955c4b60a580428ff5d8f824/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:13:59.289367 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581a3cf-d708-4a41-af88-fcb8731b4dd8 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:59.289833 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581a3cf-d708-4a41-af88-fcb8731b4dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:59.290224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581a3cf-d708-4a41-af88-fcb8731b4dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:59.331818 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-d581a3cf-d708-4a41-af88-fcb8731b4dd8 None None] MFA Rules not processed for user `eb4c352e955c4b60a580428ff5d8f824`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:13:59.333728 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-d581a3cf-d708-4a41-af88-fcb8731b4dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:59.421418 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1055/2108] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:13:59 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:13:59.439438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f1b12-fc26-43e6-9859-a6d43c5b3548 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:13:59.441752 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-aa3f1b12-fc26-43e6-9859-a6d43c5b3548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:13:59.517611 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f1b12-fc26-43e6-9859-a6d43c5b3548 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:13:59.523205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858ad15-1bd6-4ed1-855a-af4f0b31eac9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:13:59.524192 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858ad15-1bd6-4ed1-855a-af4f0b31eac9 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:13:59.524583 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858ad15-1bd6-4ed1-855a-af4f0b31eac9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:13:59.524956 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858ad15-1bd6-4ed1-855a-af4f0b31eac9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:13:59.591955 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858ad15-1bd6-4ed1-855a-af4f0b31eac9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:13:59.593799 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858ad15-1bd6-4ed1-855a-af4f0b31eac9 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:13:59.604365 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1054/2109] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:13:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:14:04.549961 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581a3cf-d708-4a41-af88-fcb8731b4dd8 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:04.551532 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-d581a3cf-d708-4a41-af88-fcb8731b4dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:04.568692 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581a3cf-d708-4a41-af88-fcb8731b4dd8 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:04.577927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355d31e-bb80-4194-aafe-cbcc04598ada service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:04.579441 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355d31e-bb80-4194-aafe-cbcc04598ada service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:04.580312 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355d31e-bb80-4194-aafe-cbcc04598ada service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:04.581099 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355d31e-bb80-4194-aafe-cbcc04598ada service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:04.584691 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f355d31e-bb80-4194-aafe-cbcc04598ada service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:04.707125 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858ad15-1bd6-4ed1-855a-af4f0b31eac9 service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:04.738241 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858ad15-1bd6-4ed1-855a-af4f0b31eac9 service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:04.749988 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88d39c-13f2-426b-8b6a-78e51dc16454 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:04.751940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88d39c-13f2-426b-8b6a-78e51dc16454 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:04.754854 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88d39c-13f2-426b-8b6a-78e51dc16454 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:04.755341 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88d39c-13f2-426b-8b6a-78e51dc16454 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:04.790985 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355d31e-bb80-4194-aafe-cbcc04598ada service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:04.793081 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355d31e-bb80-4194-aafe-cbcc04598ada service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:04.813281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1056/2110] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:14:04.854170 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:04.898539 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88d39c-13f2-426b-8b6a-78e51dc16454 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:04.900874 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88d39c-13f2-426b-8b6a-78e51dc16454 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:04.926904 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1055/2111] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:14:05.462450 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355d31e-bb80-4194-aafe-cbcc04598ada service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:05.465749 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f355d31e-bb80-4194-aafe-cbcc04598ada service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:05.478052 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:05.611721 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355d31e-bb80-4194-aafe-cbcc04598ada service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:05.620309 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbab3a0-08f3-4fbc-ad57-a29ebc1c0312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:05.621544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbab3a0-08f3-4fbc-ad57-a29ebc1c0312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:05.622137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbab3a0-08f3-4fbc-ad57-a29ebc1c0312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:05.622757 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbab3a0-08f3-4fbc-ad57-a29ebc1c0312 admin admin] PATH_INFO: `/v3/users/67b6369b61ed4c6c93b55f199b728210` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:05.625475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbab3a0-08f3-4fbc-ad57-a29ebc1c0312 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67b6369b61ed4c6c93b55f199b728210)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:05.628931 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbab3a0-08f3-4fbc-ad57-a29ebc1c0312 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:05.755718 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-fcbab3a0-08f3-4fbc-ad57-a29ebc1c0312 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67b6369b61ed4c6c93b55f199b728210'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:05.788599 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-fcbab3a0-08f3-4fbc-ad57-a29ebc1c0312 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67b6369b61ed4c6c93b55f199b728210'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:05.801180 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-fcbab3a0-08f3-4fbc-ad57-a29ebc1c0312 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67b6369b61ed4c6c93b55f199b728210'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:05.814629 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-fcbab3a0-08f3-4fbc-ad57-a29ebc1c0312 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67b6369b61ed4c6c93b55f199b728210'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:05.816060 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-fcbab3a0-08f3-4fbc-ad57-a29ebc1c0312 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67b6369b61ed4c6c93b55f199b728210'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:05.831308 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1057/2112] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:14:05 2022] DELETE /identity/v3/users/67b6369b61ed4c6c93b55f199b728210 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:05.838796 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d0fe9-5cd1-4a4e-911d-e37708927d64 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:05.839575 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d0fe9-5cd1-4a4e-911d-e37708927d64 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:05.839575 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d0fe9-5cd1-4a4e-911d-e37708927d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:05.878437 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-d32d0fe9-5cd1-4a4e-911d-e37708927d64 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:05.883081 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d32d0fe9-5cd1-4a4e-911d-e37708927d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:05.985998 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1056/2113] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:14:05.999876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbab3a0-08f3-4fbc-ad57-a29ebc1c0312 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:06.001384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-fcbab3a0-08f3-4fbc-ad57-a29ebc1c0312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:06.069091 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d0fe9-5cd1-4a4e-911d-e37708927d64 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:06.070897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d32d0fe9-5cd1-4a4e-911d-e37708927d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:06.138840 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbab3a0-08f3-4fbc-ad57-a29ebc1c0312 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:06.144578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d661f3-da2e-40e2-a964-029dda820e53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:06.145390 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d661f3-da2e-40e2-a964-029dda820e53 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:06.145827 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d661f3-da2e-40e2-a964-029dda820e53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:06.146388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d661f3-da2e-40e2-a964-029dda820e53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:06.171414 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d661f3-da2e-40e2-a964-029dda820e53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:06.175115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d661f3-da2e-40e2-a964-029dda820e53 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:06.195439 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1058/2114] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:14:06.231841 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d0fe9-5cd1-4a4e-911d-e37708927d64 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:06.239865 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fd616-316e-4d3d-a8a3-a9e455f8dd30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:06.240828 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fd616-316e-4d3d-a8a3-a9e455f8dd30 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:06.241305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fd616-316e-4d3d-a8a3-a9e455f8dd30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:06.242049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fd616-316e-4d3d-a8a3-a9e455f8dd30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:06.245279 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-7c8fd616-316e-4d3d-a8a3-a9e455f8dd30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:06.391139 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8fd616-316e-4d3d-a8a3-a9e455f8dd30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:06.399193 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8fd616-316e-4d3d-a8a3-a9e455f8dd30 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:06.429750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1057/2115] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:14:06.725454 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d661f3-da2e-40e2-a964-029dda820e53 service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:06.727428 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-19d661f3-da2e-40e2-a964-029dda820e53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:06.743766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d661f3-da2e-40e2-a964-029dda820e53 service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:06.754858 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56d160-b232-4f57-af32-54a07d3a5fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:06.756168 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56d160-b232-4f57-af32-54a07d3a5fd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:06.756851 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56d160-b232-4f57-af32-54a07d3a5fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:06.757450 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56d160-b232-4f57-af32-54a07d3a5fd8 admin admin] PATH_INFO: `/v3/projects/6f804787598449208b834ef89468713f` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:06.759694 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56d160-b232-4f57-af32-54a07d3a5fd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f804787598449208b834ef89468713f)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:06.763266 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56d160-b232-4f57-af32-54a07d3a5fd8 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:06.798214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:06.843509 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-2b56d160-b232-4f57-af32-54a07d3a5fd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f804787598449208b834ef89468713f', 'request_id': 'req-2b56d160-b232-4f57-af32-54a07d3a5fd8'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:06.850521 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-2b56d160-b232-4f57-af32-54a07d3a5fd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f804787598449208b834ef89468713f', 'request_id': 'req-2b56d160-b232-4f57-af32-54a07d3a5fd8'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:06.863163 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-2b56d160-b232-4f57-af32-54a07d3a5fd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f804787598449208b834ef89468713f', 'request_id': 'req-2b56d160-b232-4f57-af32-54a07d3a5fd8'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:06.884490 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-2b56d160-b232-4f57-af32-54a07d3a5fd8 admin admin] The token cache is being invalidate because project 6f804787598449208b834ef89468713f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:14:06.885101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-2b56d160-b232-4f57-af32-54a07d3a5fd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:06.885883 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-2b56d160-b232-4f57-af32-54a07d3a5fd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:06.889293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1059/2116] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:14:06 2022] DELETE /identity/v3/projects/6f804787598449208b834ef89468713f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:14:11.456518 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fd616-316e-4d3d-a8a3-a9e455f8dd30 service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:11.458076 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-7c8fd616-316e-4d3d-a8a3-a9e455f8dd30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:11.495132 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fd616-316e-4d3d-a8a3-a9e455f8dd30 service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:11.537436 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:11.596978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83be56-f9f8-4225-9c01-717437c44cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:11.598240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83be56-f9f8-4225-9c01-717437c44cb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:11.598807 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83be56-f9f8-4225-9c01-717437c44cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:11.599084 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83be56-f9f8-4225-9c01-717437c44cb2 admin admin] PATH_INFO: `/v3/users/cfebe82d1e014ca8bf0dc7c5f26c6045` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:11.600317 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83be56-f9f8-4225-9c01-717437c44cb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfebe82d1e014ca8bf0dc7c5f26c6045)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:11.602367 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83be56-f9f8-4225-9c01-717437c44cb2 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:11.713873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8a83be56-f9f8-4225-9c01-717437c44cb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfebe82d1e014ca8bf0dc7c5f26c6045'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:11.741373 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8a83be56-f9f8-4225-9c01-717437c44cb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfebe82d1e014ca8bf0dc7c5f26c6045'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:11.769551 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8a83be56-f9f8-4225-9c01-717437c44cb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfebe82d1e014ca8bf0dc7c5f26c6045'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:11.793806 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8a83be56-f9f8-4225-9c01-717437c44cb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfebe82d1e014ca8bf0dc7c5f26c6045'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:11.796673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8a83be56-f9f8-4225-9c01-717437c44cb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfebe82d1e014ca8bf0dc7c5f26c6045'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:11.808957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1058/2117] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:14:11 2022] DELETE /identity/v3/users/cfebe82d1e014ca8bf0dc7c5f26c6045 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:11.818166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c071e81-7e51-4c33-b368-ba5800f5d052 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:11.819520 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c071e81-7e51-4c33-b368-ba5800f5d052 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:11.819520 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c071e81-7e51-4c33-b368-ba5800f5d052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:11.855627 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-8c071e81-7e51-4c33-b368-ba5800f5d052 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:11.856851 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-8c071e81-7e51-4c33-b368-ba5800f5d052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:11.956201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1060/2118] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:11.968724 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83be56-f9f8-4225-9c01-717437c44cb2 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:11.970782 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-8a83be56-f9f8-4225-9c01-717437c44cb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:12.080611 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83be56-f9f8-4225-9c01-717437c44cb2 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:12.087737 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b354625-1b9a-46df-8965-2633e7766f0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:12.088785 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b354625-1b9a-46df-8965-2633e7766f0c service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:12.089144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b354625-1b9a-46df-8965-2633e7766f0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:12.089590 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b354625-1b9a-46df-8965-2633e7766f0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:12.105015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b354625-1b9a-46df-8965-2633e7766f0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:12.108243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b354625-1b9a-46df-8965-2633e7766f0c service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:12.121970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1059/2119] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:14:12.416298 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c071e81-7e51-4c33-b368-ba5800f5d052 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:12.417911 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-8c071e81-7e51-4c33-b368-ba5800f5d052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:12.433279 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c071e81-7e51-4c33-b368-ba5800f5d052 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:12.452774 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c955102-778d-44e3-90a9-ca4ffc1bc1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:12.453432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c955102-778d-44e3-90a9-ca4ffc1bc1a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:12.453733 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c955102-778d-44e3-90a9-ca4ffc1bc1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:12.454020 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c955102-778d-44e3-90a9-ca4ffc1bc1a6 admin admin] PATH_INFO: `/v3/projects/146824c806c14bd2b68b149bda56e2b8` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:12.455206 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c955102-778d-44e3-90a9-ca4ffc1bc1a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=146824c806c14bd2b68b149bda56e2b8)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:12.457262 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c955102-778d-44e3-90a9-ca4ffc1bc1a6 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:12.528189 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-4c955102-778d-44e3-90a9-ca4ffc1bc1a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '146824c806c14bd2b68b149bda56e2b8', 'request_id': 'req-4c955102-778d-44e3-90a9-ca4ffc1bc1a6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:12.545899 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-4c955102-778d-44e3-90a9-ca4ffc1bc1a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '146824c806c14bd2b68b149bda56e2b8', 'request_id': 'req-4c955102-778d-44e3-90a9-ca4ffc1bc1a6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:12.559813 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-4c955102-778d-44e3-90a9-ca4ffc1bc1a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '146824c806c14bd2b68b149bda56e2b8', 'request_id': 'req-4c955102-778d-44e3-90a9-ca4ffc1bc1a6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:12.577592 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-4c955102-778d-44e3-90a9-ca4ffc1bc1a6 admin admin] The token cache is being invalidate because project 146824c806c14bd2b68b149bda56e2b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:14:12.577993 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-4c955102-778d-44e3-90a9-ca4ffc1bc1a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:12.578766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-4c955102-778d-44e3-90a9-ca4ffc1bc1a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:12.584342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1061/2120] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:14:12 2022] DELETE /identity/v3/projects/146824c806c14bd2b68b149bda56e2b8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:12.586050 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b354625-1b9a-46df-8965-2633e7766f0c service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:12.595433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b354625-1b9a-46df-8965-2633e7766f0c service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:12.673100 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:12.680613 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9666822-137a-45bc-b0c9-a5a98d09c95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:12.681620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9666822-137a-45bc-b0c9-a5a98d09c95d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:12.682051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9666822-137a-45bc-b0c9-a5a98d09c95d admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:12.684338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9666822-137a-45bc-b0c9-a5a98d09c95d admin admin] PATH_INFO: `/v3/users/62a0003f31484004a6bff502910f066d` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:12.685824 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9666822-137a-45bc-b0c9-a5a98d09c95d admin admin] RBAC: Authorizing `identity:delete_user(user_id=62a0003f31484004a6bff502910f066d)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:12.689227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9666822-137a-45bc-b0c9-a5a98d09c95d admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:12.759511 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a9666822-137a-45bc-b0c9-a5a98d09c95d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62a0003f31484004a6bff502910f066d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:12.768869 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a9666822-137a-45bc-b0c9-a5a98d09c95d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62a0003f31484004a6bff502910f066d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:12.784400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a9666822-137a-45bc-b0c9-a5a98d09c95d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62a0003f31484004a6bff502910f066d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:12.791970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a9666822-137a-45bc-b0c9-a5a98d09c95d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62a0003f31484004a6bff502910f066d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:12.792888 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a9666822-137a-45bc-b0c9-a5a98d09c95d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62a0003f31484004a6bff502910f066d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:12.799834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1060/2121] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:14:12 2022] DELETE /identity/v3/users/62a0003f31484004a6bff502910f066d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:13.124001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c955102-778d-44e3-90a9-ca4ffc1bc1a6 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:13.125639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-4c955102-778d-44e3-90a9-ca4ffc1bc1a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:13.146974 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c955102-778d-44e3-90a9-ca4ffc1bc1a6 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:13.182769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:13.212208 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e522836-188b-42f4-80e2-3577b18cd275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:13.212850 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e522836-188b-42f4-80e2-3577b18cd275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:13.213126 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e522836-188b-42f4-80e2-3577b18cd275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:13.213432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e522836-188b-42f4-80e2-3577b18cd275 admin admin] PATH_INFO: `/v3/projects/1727f4ea13024749b0266b53777ea8cd` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:13.215190 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e522836-188b-42f4-80e2-3577b18cd275 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1727f4ea13024749b0266b53777ea8cd)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:13.217648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e522836-188b-42f4-80e2-3577b18cd275 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:13.270444 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-5e522836-188b-42f4-80e2-3577b18cd275 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1727f4ea13024749b0266b53777ea8cd', 'request_id': 'req-5e522836-188b-42f4-80e2-3577b18cd275'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:13.277485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-5e522836-188b-42f4-80e2-3577b18cd275 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1727f4ea13024749b0266b53777ea8cd', 'request_id': 'req-5e522836-188b-42f4-80e2-3577b18cd275'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:13.288160 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-5e522836-188b-42f4-80e2-3577b18cd275 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1727f4ea13024749b0266b53777ea8cd', 'request_id': 'req-5e522836-188b-42f4-80e2-3577b18cd275'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:13.302343 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-5e522836-188b-42f4-80e2-3577b18cd275 admin admin] The token cache is being invalidate because project 1727f4ea13024749b0266b53777ea8cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:14:13.302645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-5e522836-188b-42f4-80e2-3577b18cd275 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:13.303315 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-5e522836-188b-42f4-80e2-3577b18cd275 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:13.304833 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1062/2122] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:14:13 2022] DELETE /identity/v3/projects/1727f4ea13024749b0266b53777ea8cd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:14:13.311375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08336395-5aa9-4697-b339-65e3fc9d1504 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:13.311638 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08336395-5aa9-4697-b339-65e3fc9d1504 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:13.311916 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08336395-5aa9-4697-b339-65e3fc9d1504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:13.335331 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-08336395-5aa9-4697-b339-65e3fc9d1504 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:13.338590 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-08336395-5aa9-4697-b339-65e3fc9d1504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:13.408020 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1061/2123] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:13.418156 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7d080-c455-4394-9b10-43cef86a253c None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:13.418557 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7d080-c455-4394-9b10-43cef86a253c None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:13.418833 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7d080-c455-4394-9b10-43cef86a253c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:13.439038 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-16e7d080-c455-4394-9b10-43cef86a253c None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:13.440162 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-16e7d080-c455-4394-9b10-43cef86a253c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:13.448561 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1063/2124] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:13.453920 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08336395-5aa9-4697-b339-65e3fc9d1504 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:13.473747 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08336395-5aa9-4697-b339-65e3fc9d1504 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:13.480084 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d030a-b16b-4d7e-92f0-46cc8347254c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:13.480825 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d030a-b16b-4d7e-92f0-46cc8347254c admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:13.481144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d030a-b16b-4d7e-92f0-46cc8347254c admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:13.481455 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d030a-b16b-4d7e-92f0-46cc8347254c admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:13.482556 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d030a-b16b-4d7e-92f0-46cc8347254c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:14:13.483079 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d030a-b16b-4d7e-92f0-46cc8347254c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:13.484704 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d030a-b16b-4d7e-92f0-46cc8347254c admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:13.497458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1062/2125] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:14:13.502678 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7d080-c455-4394-9b10-43cef86a253c None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:13.516565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7d080-c455-4394-9b10-43cef86a253c None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:13.521475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b52fd-3bc8-462b-be4e-d0f25c43971a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:13.522143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b52fd-3bc8-462b-be4e-d0f25c43971a admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:13.522429 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b52fd-3bc8-462b-be4e-d0f25c43971a admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:13.524024 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b52fd-3bc8-462b-be4e-d0f25c43971a admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:13.525512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b52fd-3bc8-462b-be4e-d0f25c43971a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:13.530448 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b52fd-3bc8-462b-be4e-d0f25c43971a admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:13.573205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1064/2126] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:14:13 2022] POST /identity/v3/projects => generated 375 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:14:13.577487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d030a-b16b-4d7e-92f0-46cc8347254c admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:13.588223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d030a-b16b-4d7e-92f0-46cc8347254c admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:13.649232 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc960f2-c4a1-41a4-934c-c8a8bcb840b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:13.649901 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc960f2-c4a1-41a4-934c-c8a8bcb840b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:13.650278 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc960f2-c4a1-41a4-934c-c8a8bcb840b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:13.650492 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc960f2-c4a1-41a4-934c-c8a8bcb840b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:13.651144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc960f2-c4a1-41a4-934c-c8a8bcb840b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:13.653114 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc960f2-c4a1-41a4-934c-c8a8bcb840b0 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:13.662715 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:13.680550 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1063/2127] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:14:13 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:14:13.682154 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b52fd-3bc8-462b-be4e-d0f25c43971a admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:13.693993 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b52fd-3bc8-462b-be4e-d0f25c43971a admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:13.699103 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eaf824-62b0-40e8-a99c-ad0b12de6178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:13.699719 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eaf824-62b0-40e8-a99c-ad0b12de6178 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:13.700000 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eaf824-62b0-40e8-a99c-ad0b12de6178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:13.700275 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eaf824-62b0-40e8-a99c-ad0b12de6178 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:13.701141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eaf824-62b0-40e8-a99c-ad0b12de6178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:13.702644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eaf824-62b0-40e8-a99c-ad0b12de6178 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:13.713666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1065/2128] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:13.718811 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc960f2-c4a1-41a4-934c-c8a8bcb840b0 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:13.731462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc960f2-c4a1-41a4-934c-c8a8bcb840b0 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:13.738512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517c23c-3918-4dbb-b535-4b0b39bdfda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:13.739117 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517c23c-3918-4dbb-b535-4b0b39bdfda6 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:13.739399 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517c23c-3918-4dbb-b535-4b0b39bdfda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:13.739677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517c23c-3918-4dbb-b535-4b0b39bdfda6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:13.740331 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517c23c-3918-4dbb-b535-4b0b39bdfda6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:13.742019 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517c23c-3918-4dbb-b535-4b0b39bdfda6 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:13.754502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1064/2129] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:13.758807 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eaf824-62b0-40e8-a99c-ad0b12de6178 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:13.771317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eaf824-62b0-40e8-a99c-ad0b12de6178 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:13.776636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3413453-6379-41ad-82e7-a602135f2c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:13.777405 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3413453-6379-41ad-82e7-a602135f2c63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:13.777789 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3413453-6379-41ad-82e7-a602135f2c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:13.778070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3413453-6379-41ad-82e7-a602135f2c63 admin admin] PATH_INFO: `/v3/projects/f4b85d533b2249248a1d488f6d4fd19a/users/a90b91aa4abe48d39ee7373bcfb6c339/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:13.799126 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3413453-6379-41ad-82e7-a602135f2c63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b85d533b2249248a1d488f6d4fd19a, user_id=a90b91aa4abe48d39ee7373bcfb6c339, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:13.802005 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3413453-6379-41ad-82e7-a602135f2c63 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:13.817891 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1066/2130] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:14:13 2022] PUT /identity/v3/projects/f4b85d533b2249248a1d488f6d4fd19a/users/a90b91aa4abe48d39ee7373bcfb6c339/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:13.831072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba5f4b-79cf-4c1c-8671-ee5a70f291c6 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:13.831684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba5f4b-79cf-4c1c-8671-ee5a70f291c6 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:13.832038 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba5f4b-79cf-4c1c-8671-ee5a70f291c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:13.867016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-dcba5f4b-79cf-4c1c-8671-ee5a70f291c6 None None] MFA Rules not processed for user `a90b91aa4abe48d39ee7373bcfb6c339`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:13.868303 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-dcba5f4b-79cf-4c1c-8671-ee5a70f291c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:13.946342 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1065/2131] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:13.947836 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3413453-6379-41ad-82e7-a602135f2c63 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:14.032049 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3413453-6379-41ad-82e7-a602135f2c63 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:14.037150 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69717a9-835a-4f56-ac12-297ab7dab046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:14.037782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69717a9-835a-4f56-ac12-297ab7dab046 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:14.038064 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69717a9-835a-4f56-ac12-297ab7dab046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:14.038330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69717a9-835a-4f56-ac12-297ab7dab046 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:14.038954 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69717a9-835a-4f56-ac12-297ab7dab046 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:14.040864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69717a9-835a-4f56-ac12-297ab7dab046 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:14.046769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:14.073864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1067/2132] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:14:13 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:14:14.083213 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba5f4b-79cf-4c1c-8671-ee5a70f291c6 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:14.097398 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba5f4b-79cf-4c1c-8671-ee5a70f291c6 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:14.158273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ec320-f0f8-4a67-89be-a9ddf0e6c468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:14.158896 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ec320-f0f8-4a67-89be-a9ddf0e6c468 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:14.159203 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ec320-f0f8-4a67-89be-a9ddf0e6c468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:14.159479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ec320-f0f8-4a67-89be-a9ddf0e6c468 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:14.160118 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ec320-f0f8-4a67-89be-a9ddf0e6c468 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:14.162083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ec320-f0f8-4a67-89be-a9ddf0e6c468 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:14.187416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1066/2133] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:14:14 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:14:14.191857 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69717a9-835a-4f56-ac12-297ab7dab046 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:14.203035 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69717a9-835a-4f56-ac12-297ab7dab046 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:14.207827 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1d65d-3d6d-4c80-bac0-a85843a8a436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:14.208442 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1d65d-3d6d-4c80-bac0-a85843a8a436 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:14.208724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1d65d-3d6d-4c80-bac0-a85843a8a436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:14.209000 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1d65d-3d6d-4c80-bac0-a85843a8a436 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:14.209659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1d65d-3d6d-4c80-bac0-a85843a8a436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:14.211366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1d65d-3d6d-4c80-bac0-a85843a8a436 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:14.222409 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1068/2134] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:14.227851 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ec320-f0f8-4a67-89be-a9ddf0e6c468 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:14.239043 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ec320-f0f8-4a67-89be-a9ddf0e6c468 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:14.246181 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b015e-fbbb-4495-89dd-0df4f6f2d60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:14.247072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b015e-fbbb-4495-89dd-0df4f6f2d60d admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:14.247352 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b015e-fbbb-4495-89dd-0df4f6f2d60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:14.247629 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b015e-fbbb-4495-89dd-0df4f6f2d60d admin admin] PATH_INFO: `/v3/projects/bffbf4fc8c5b497da6b9b93dcdeb3ab5/users/973c01e17dba4b29a7af89a200d2955e/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:14.264630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b015e-fbbb-4495-89dd-0df4f6f2d60d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bffbf4fc8c5b497da6b9b93dcdeb3ab5, user_id=973c01e17dba4b29a7af89a200d2955e, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:14.267356 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b015e-fbbb-4495-89dd-0df4f6f2d60d admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:14.282196 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1067/2135] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:14:14 2022] PUT /identity/v3/projects/bffbf4fc8c5b497da6b9b93dcdeb3ab5/users/973c01e17dba4b29a7af89a200d2955e/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:14.286488 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1d65d-3d6d-4c80-bac0-a85843a8a436 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:14.297546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1d65d-3d6d-4c80-bac0-a85843a8a436 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:14.358962 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbb077-c7dc-450b-9de6-a5bacb88ebc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:14.359663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbb077-c7dc-450b-9de6-a5bacb88ebc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:14.359935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbb077-c7dc-450b-9de6-a5bacb88ebc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:14.360207 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbb077-c7dc-450b-9de6-a5bacb88ebc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:14.360903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dbb077-c7dc-450b-9de6-a5bacb88ebc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:14.362530 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dbb077-c7dc-450b-9de6-a5bacb88ebc5 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:14.373804 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1069/2136] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:14.378332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b015e-fbbb-4495-89dd-0df4f6f2d60d admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:14.390137 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b015e-fbbb-4495-89dd-0df4f6f2d60d admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:14.395384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16280f44-9264-4b40-8a80-3a29ab5ee5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:14.396068 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16280f44-9264-4b40-8a80-3a29ab5ee5e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:14.396349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16280f44-9264-4b40-8a80-3a29ab5ee5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:14.396625 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16280f44-9264-4b40-8a80-3a29ab5ee5e0 admin admin] PATH_INFO: `/v3/projects/bffbf4fc8c5b497da6b9b93dcdeb3ab5/users/973c01e17dba4b29a7af89a200d2955e/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:14.398890 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16280f44-9264-4b40-8a80-3a29ab5ee5e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bffbf4fc8c5b497da6b9b93dcdeb3ab5, user_id=973c01e17dba4b29a7af89a200d2955e, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:14.401567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16280f44-9264-4b40-8a80-3a29ab5ee5e0 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:14.413327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1068/2137] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:14:14 2022] PUT /identity/v3/projects/bffbf4fc8c5b497da6b9b93dcdeb3ab5/users/973c01e17dba4b29a7af89a200d2955e/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:14.424545 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229b10a-680e-4bb1-aec5-beea43c77a54 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:14.424861 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229b10a-680e-4bb1-aec5-beea43c77a54 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:14.425148 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229b10a-680e-4bb1-aec5-beea43c77a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:14.456989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-7229b10a-680e-4bb1-aec5-beea43c77a54 None None] MFA Rules not processed for user `973c01e17dba4b29a7af89a200d2955e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:14.458173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-7229b10a-680e-4bb1-aec5-beea43c77a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:14.533919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1070/2138] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:14.546437 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16280f44-9264-4b40-8a80-3a29ab5ee5e0 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:14.548218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-16280f44-9264-4b40-8a80-3a29ab5ee5e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:14.578040 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:14.616737 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16280f44-9264-4b40-8a80-3a29ab5ee5e0 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:14.620708 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88edfed-1244-43d7-94be-fc116146c6d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:14.621335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88edfed-1244-43d7-94be-fc116146c6d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:14.621607 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88edfed-1244-43d7-94be-fc116146c6d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:14.621905 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88edfed-1244-43d7-94be-fc116146c6d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:14.685173 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88edfed-1244-43d7-94be-fc116146c6d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:14.686667 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88edfed-1244-43d7-94be-fc116146c6d2 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:14.696509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1069/2139] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:14:20.411982 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229b10a-680e-4bb1-aec5-beea43c77a54 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:20.419203 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-7229b10a-680e-4bb1-aec5-beea43c77a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:20.551210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:20.558955 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229b10a-680e-4bb1-aec5-beea43c77a54 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:20.566782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661814b0-634e-40d9-85c5-627aba7ecd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:20.567823 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661814b0-634e-40d9-85c5-627aba7ecd93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:20.568306 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661814b0-634e-40d9-85c5-627aba7ecd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:20.568734 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661814b0-634e-40d9-85c5-627aba7ecd93 admin admin] PATH_INFO: `/v3/users/c2e1753f05db4a1b8a2dd00041aa09a7` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:20.570766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661814b0-634e-40d9-85c5-627aba7ecd93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2e1753f05db4a1b8a2dd00041aa09a7)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:20.574197 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661814b0-634e-40d9-85c5-627aba7ecd93 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:20.659953 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-661814b0-634e-40d9-85c5-627aba7ecd93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2e1753f05db4a1b8a2dd00041aa09a7'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:20.681425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-661814b0-634e-40d9-85c5-627aba7ecd93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2e1753f05db4a1b8a2dd00041aa09a7'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:20.701299 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-661814b0-634e-40d9-85c5-627aba7ecd93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2e1753f05db4a1b8a2dd00041aa09a7'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:20.709898 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88edfed-1244-43d7-94be-fc116146c6d2 service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:20.737959 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-661814b0-634e-40d9-85c5-627aba7ecd93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2e1753f05db4a1b8a2dd00041aa09a7'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:20.739388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-661814b0-634e-40d9-85c5-627aba7ecd93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2e1753f05db4a1b8a2dd00041aa09a7'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:20.740162 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88edfed-1244-43d7-94be-fc116146c6d2 service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:20.752826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1071/2140] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:14:20 2022] DELETE /identity/v3/users/c2e1753f05db4a1b8a2dd00041aa09a7 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:20.753813 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa7fd9-332a-4254-875d-c65ad40c5e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:20.755171 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa7fd9-332a-4254-875d-c65ad40c5e11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:20.755860 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa7fd9-332a-4254-875d-c65ad40c5e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:20.756530 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa7fd9-332a-4254-875d-c65ad40c5e11 admin admin] PATH_INFO: `/v3/users/a90b91aa4abe48d39ee7373bcfb6c339` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:20.757401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec545e-ab11-4c44-9b48-f8fb16a15147 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:20.757687 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec545e-ab11-4c44-9b48-f8fb16a15147 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:20.757982 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec545e-ab11-4c44-9b48-f8fb16a15147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:20.758511 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa7fd9-332a-4254-875d-c65ad40c5e11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a90b91aa4abe48d39ee7373bcfb6c339)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:20.760962 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa7fd9-332a-4254-875d-c65ad40c5e11 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:20.805055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-58ec545e-ab11-4c44-9b48-f8fb16a15147 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:20.806394 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-58ec545e-ab11-4c44-9b48-f8fb16a15147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:20.865846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c7fa7fd9-332a-4254-875d-c65ad40c5e11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a90b91aa4abe48d39ee7373bcfb6c339'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:20.889565 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c7fa7fd9-332a-4254-875d-c65ad40c5e11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a90b91aa4abe48d39ee7373bcfb6c339'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:20.921842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c7fa7fd9-332a-4254-875d-c65ad40c5e11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a90b91aa4abe48d39ee7373bcfb6c339'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:20.936469 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c7fa7fd9-332a-4254-875d-c65ad40c5e11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a90b91aa4abe48d39ee7373bcfb6c339'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:20.938081 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c7fa7fd9-332a-4254-875d-c65ad40c5e11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a90b91aa4abe48d39ee7373bcfb6c339'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:20.940815 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1072/2141] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:20.952200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec545e-ab11-4c44-9b48-f8fb16a15147 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:20.957088 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1070/2142] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:14:20 2022] DELETE /identity/v3/users/a90b91aa4abe48d39ee7373bcfb6c339 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:20.959121 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-58ec545e-ab11-4c44-9b48-f8fb16a15147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:20.960480 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d4ca9-03ae-4211-9cd4-636d8859eea0 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:20.960921 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d4ca9-03ae-4211-9cd4-636d8859eea0 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:20.961296 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d4ca9-03ae-4211-9cd4-636d8859eea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:21.000191 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-e37d4ca9-03ae-4211-9cd4-636d8859eea0 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:21.001890 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-e37d4ca9-03ae-4211-9cd4-636d8859eea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:21.087788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:21.120937 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec545e-ab11-4c44-9b48-f8fb16a15147 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:21.143499 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1071/2143] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:21.191280 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d4ca9-03ae-4211-9cd4-636d8859eea0 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:21.196423 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-e37d4ca9-03ae-4211-9cd4-636d8859eea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:21.217689 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:21.271650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3ba17-9f37-4dad-a854-3f7037fa5931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:21.273225 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3ba17-9f37-4dad-a854-3f7037fa5931 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:21.274391 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3ba17-9f37-4dad-a854-3f7037fa5931 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:21.275296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3ba17-9f37-4dad-a854-3f7037fa5931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:21.282843 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-abc3ba17-9f37-4dad-a854-3f7037fa5931 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:21.312162 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3ba17-9f37-4dad-a854-3f7037fa5931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:21.314988 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3ba17-9f37-4dad-a854-3f7037fa5931 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:21.340469 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1073/2144] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:14:21.343534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d4ca9-03ae-4211-9cd4-636d8859eea0 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:21.349197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712abe00-7e62-4313-8cd0-dcccc6fd74d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:21.350059 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712abe00-7e62-4313-8cd0-dcccc6fd74d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:21.350452 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712abe00-7e62-4313-8cd0-dcccc6fd74d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:21.350834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712abe00-7e62-4313-8cd0-dcccc6fd74d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:21.362676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712abe00-7e62-4313-8cd0-dcccc6fd74d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:21.364427 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712abe00-7e62-4313-8cd0-dcccc6fd74d5 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:21.381412 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1072/2145] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:14:22.325744 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3ba17-9f37-4dad-a854-3f7037fa5931 service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:22.328759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-abc3ba17-9f37-4dad-a854-3f7037fa5931 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:22.350021 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3ba17-9f37-4dad-a854-3f7037fa5931 service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:22.359988 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec6c34-5938-4f53-9fb5-79a98cf901a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:22.360223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712abe00-7e62-4313-8cd0-dcccc6fd74d5 service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:22.361334 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec6c34-5938-4f53-9fb5-79a98cf901a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:22.361552 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec6c34-5938-4f53-9fb5-79a98cf901a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:22.361763 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-712abe00-7e62-4313-8cd0-dcccc6fd74d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:22.361934 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec6c34-5938-4f53-9fb5-79a98cf901a9 admin admin] PATH_INFO: `/v3/projects/88723982e96f47c68bb1ad26a49ddece` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:22.363792 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec6c34-5938-4f53-9fb5-79a98cf901a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88723982e96f47c68bb1ad26a49ddece)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:22.366311 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec6c34-5938-4f53-9fb5-79a98cf901a9 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:22.382453 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712abe00-7e62-4313-8cd0-dcccc6fd74d5 service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:22.390599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadb00e8-0bda-466d-97b5-28c37dbeeb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:22.391606 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb00e8-0bda-466d-97b5-28c37dbeeb3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:22.391702 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb00e8-0bda-466d-97b5-28c37dbeeb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:22.392106 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb00e8-0bda-466d-97b5-28c37dbeeb3d admin admin] PATH_INFO: `/v3/projects/f4b85d533b2249248a1d488f6d4fd19a` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:22.395144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadb00e8-0bda-466d-97b5-28c37dbeeb3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4b85d533b2249248a1d488f6d4fd19a)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:22.397216 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadb00e8-0bda-466d-97b5-28c37dbeeb3d admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:22.477817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-75ec6c34-5938-4f53-9fb5-79a98cf901a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88723982e96f47c68bb1ad26a49ddece', 'request_id': 'req-75ec6c34-5938-4f53-9fb5-79a98cf901a9'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:22.492219 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-75ec6c34-5938-4f53-9fb5-79a98cf901a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88723982e96f47c68bb1ad26a49ddece', 'request_id': 'req-75ec6c34-5938-4f53-9fb5-79a98cf901a9'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:22.492684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-eadb00e8-0bda-466d-97b5-28c37dbeeb3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4b85d533b2249248a1d488f6d4fd19a', 'request_id': 'req-eadb00e8-0bda-466d-97b5-28c37dbeeb3d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:22.503213 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-eadb00e8-0bda-466d-97b5-28c37dbeeb3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4b85d533b2249248a1d488f6d4fd19a', 'request_id': 'req-eadb00e8-0bda-466d-97b5-28c37dbeeb3d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:22.513087 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-75ec6c34-5938-4f53-9fb5-79a98cf901a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88723982e96f47c68bb1ad26a49ddece', 'request_id': 'req-75ec6c34-5938-4f53-9fb5-79a98cf901a9'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:22.516103 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-eadb00e8-0bda-466d-97b5-28c37dbeeb3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4b85d533b2249248a1d488f6d4fd19a', 'request_id': 'req-eadb00e8-0bda-466d-97b5-28c37dbeeb3d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:22.535222 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-eadb00e8-0bda-466d-97b5-28c37dbeeb3d admin admin] The token cache is being invalidate because project f4b85d533b2249248a1d488f6d4fd19a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:14:22.535676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-eadb00e8-0bda-466d-97b5-28c37dbeeb3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:22.537984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-75ec6c34-5938-4f53-9fb5-79a98cf901a9 admin admin] The token cache is being invalidate because project 88723982e96f47c68bb1ad26a49ddece was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:14:22.538196 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-eadb00e8-0bda-466d-97b5-28c37dbeeb3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:22.538321 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-75ec6c34-5938-4f53-9fb5-79a98cf901a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:22.543361 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-75ec6c34-5938-4f53-9fb5-79a98cf901a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:22.543746 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1073/2146] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:14:22 2022] DELETE /identity/v3/projects/f4b85d533b2249248a1d488f6d4fd19a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:14:22.545149 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadb00e8-0bda-466d-97b5-28c37dbeeb3d admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:22.546769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1074/2147] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:14:22 2022] DELETE /identity/v3/projects/88723982e96f47c68bb1ad26a49ddece => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:22.547793 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-eadb00e8-0bda-466d-97b5-28c37dbeeb3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:22.552276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec6c34-5938-4f53-9fb5-79a98cf901a9 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:22.558631 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-75ec6c34-5938-4f53-9fb5-79a98cf901a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:22.666563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadb00e8-0bda-466d-97b5-28c37dbeeb3d admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:22.682875 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd0f8d-9c91-465f-9bd7-8a496356d02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:22.683800 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd0f8d-9c91-465f-9bd7-8a496356d02d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:22.684223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd0f8d-9c91-465f-9bd7-8a496356d02d admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:22.684526 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd0f8d-9c91-465f-9bd7-8a496356d02d admin admin] PATH_INFO: `/v3/users/973c01e17dba4b29a7af89a200d2955e` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:22.687675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd0f8d-9c91-465f-9bd7-8a496356d02d admin admin] RBAC: Authorizing `identity:delete_user(user_id=973c01e17dba4b29a7af89a200d2955e)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:22.690536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd0f8d-9c91-465f-9bd7-8a496356d02d admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:22.720431 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:22.748295 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec6c34-5938-4f53-9fb5-79a98cf901a9 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:22.760756 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca72d2c-7c51-45d6-a947-3b98d2c0a019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:22.761427 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca72d2c-7c51-45d6-a947-3b98d2c0a019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:22.762120 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca72d2c-7c51-45d6-a947-3b98d2c0a019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:22.762420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca72d2c-7c51-45d6-a947-3b98d2c0a019 admin admin] PATH_INFO: `/v3/users/6ed0078f192045bab2573723c3c666d2` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:22.766613 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca72d2c-7c51-45d6-a947-3b98d2c0a019 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ed0078f192045bab2573723c3c666d2)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:22.770164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca72d2c-7c51-45d6-a947-3b98d2c0a019 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:22.774017 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-7fcd0f8d-9c91-465f-9bd7-8a496356d02d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '973c01e17dba4b29a7af89a200d2955e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:22.789780 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-7fcd0f8d-9c91-465f-9bd7-8a496356d02d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '973c01e17dba4b29a7af89a200d2955e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:22.812894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-7fcd0f8d-9c91-465f-9bd7-8a496356d02d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '973c01e17dba4b29a7af89a200d2955e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:22.824820 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-7fcd0f8d-9c91-465f-9bd7-8a496356d02d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '973c01e17dba4b29a7af89a200d2955e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:22.827384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-7fcd0f8d-9c91-465f-9bd7-8a496356d02d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '973c01e17dba4b29a7af89a200d2955e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:22.835472 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1074/2148] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:14:22 2022] DELETE /identity/v3/users/973c01e17dba4b29a7af89a200d2955e => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:22.900309 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-dca72d2c-7c51-45d6-a947-3b98d2c0a019 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ed0078f192045bab2573723c3c666d2'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:22.917304 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:22.937890 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-dca72d2c-7c51-45d6-a947-3b98d2c0a019 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ed0078f192045bab2573723c3c666d2'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:22.960975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-dca72d2c-7c51-45d6-a947-3b98d2c0a019 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ed0078f192045bab2573723c3c666d2'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:22.976914 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-dca72d2c-7c51-45d6-a947-3b98d2c0a019 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ed0078f192045bab2573723c3c666d2'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:22.978182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-dca72d2c-7c51-45d6-a947-3b98d2c0a019 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ed0078f192045bab2573723c3c666d2'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:22.995944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1075/2149] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:14:22 2022] DELETE /identity/v3/users/6ed0078f192045bab2573723c3c666d2 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:23.457253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd0f8d-9c91-465f-9bd7-8a496356d02d admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:23.458931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-7fcd0f8d-9c91-465f-9bd7-8a496356d02d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:23.524243 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca72d2c-7c51-45d6-a947-3b98d2c0a019 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:23.526898 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-dca72d2c-7c51-45d6-a947-3b98d2c0a019 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:23.555351 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd0f8d-9c91-465f-9bd7-8a496356d02d admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:23.647153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca72d2c-7c51-45d6-a947-3b98d2c0a019 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:23.647976 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff29873-d351-4bab-9d48-702eaa3e1a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:23.648634 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff29873-d351-4bab-9d48-702eaa3e1a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:23.649177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff29873-d351-4bab-9d48-702eaa3e1a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:23.649447 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff29873-d351-4bab-9d48-702eaa3e1a0b admin admin] PATH_INFO: `/v3/projects/61d68ea03ca24388b663ed895d5fe9d8` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:23.655631 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff29873-d351-4bab-9d48-702eaa3e1a0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=61d68ea03ca24388b663ed895d5fe9d8)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:23.655631 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff29873-d351-4bab-9d48-702eaa3e1a0b admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:23.656164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5254d21-3c09-4801-bd40-36e488b808c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:23.657182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5254d21-3c09-4801-bd40-36e488b808c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:23.658729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5254d21-3c09-4801-bd40-36e488b808c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:23.659886 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5254d21-3c09-4801-bd40-36e488b808c0 admin admin] PATH_INFO: `/v3/projects/bffbf4fc8c5b497da6b9b93dcdeb3ab5` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:23.661603 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5254d21-3c09-4801-bd40-36e488b808c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bffbf4fc8c5b497da6b9b93dcdeb3ab5)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:23.665412 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5254d21-3c09-4801-bd40-36e488b808c0 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:23.738856 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:23.749522 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9ff29873-d351-4bab-9d48-702eaa3e1a0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61d68ea03ca24388b663ed895d5fe9d8', 'request_id': 'req-9ff29873-d351-4bab-9d48-702eaa3e1a0b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:23.753004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e5254d21-3c09-4801-bd40-36e488b808c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bffbf4fc8c5b497da6b9b93dcdeb3ab5', 'request_id': 'req-e5254d21-3c09-4801-bd40-36e488b808c0'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:23.761361 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e5254d21-3c09-4801-bd40-36e488b808c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bffbf4fc8c5b497da6b9b93dcdeb3ab5', 'request_id': 'req-e5254d21-3c09-4801-bd40-36e488b808c0'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:23.762479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9ff29873-d351-4bab-9d48-702eaa3e1a0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61d68ea03ca24388b663ed895d5fe9d8', 'request_id': 'req-9ff29873-d351-4bab-9d48-702eaa3e1a0b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:23.774939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9ff29873-d351-4bab-9d48-702eaa3e1a0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61d68ea03ca24388b663ed895d5fe9d8', 'request_id': 'req-9ff29873-d351-4bab-9d48-702eaa3e1a0b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:23.775694 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e5254d21-3c09-4801-bd40-36e488b808c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bffbf4fc8c5b497da6b9b93dcdeb3ab5', 'request_id': 'req-e5254d21-3c09-4801-bd40-36e488b808c0'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:23.793967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e5254d21-3c09-4801-bd40-36e488b808c0 admin admin] The token cache is being invalidate because project bffbf4fc8c5b497da6b9b93dcdeb3ab5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:14:23.794918 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e5254d21-3c09-4801-bd40-36e488b808c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:23.797506 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e5254d21-3c09-4801-bd40-36e488b808c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:23.800356 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1076/2150] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:14:23 2022] DELETE /identity/v3/projects/bffbf4fc8c5b497da6b9b93dcdeb3ab5 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:23.809542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e27305-561e-4feb-8ceb-08e08836ed1f None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:23.810272 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e27305-561e-4feb-8ceb-08e08836ed1f None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:23.811970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e27305-561e-4feb-8ceb-08e08836ed1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:23.815474 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9ff29873-d351-4bab-9d48-702eaa3e1a0b admin admin] The token cache is being invalidate because project 61d68ea03ca24388b663ed895d5fe9d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:14:23.815943 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9ff29873-d351-4bab-9d48-702eaa3e1a0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:23.817160 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9ff29873-d351-4bab-9d48-702eaa3e1a0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:23.819949 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1075/2151] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:14:23 2022] DELETE /identity/v3/projects/61d68ea03ca24388b663ed895d5fe9d8 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:23.825182 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff29873-d351-4bab-9d48-702eaa3e1a0b admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:23.832723 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9ff29873-d351-4bab-9d48-702eaa3e1a0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:23.858824 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-06e27305-561e-4feb-8ceb-08e08836ed1f None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:23.861414 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-06e27305-561e-4feb-8ceb-08e08836ed1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:23.970021 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1077/2152] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:14:23.976393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc49f0c-d280-4c73-a8ae-49e62f1db976 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:23.976686 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc49f0c-d280-4c73-a8ae-49e62f1db976 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:23.976931 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc49f0c-d280-4c73-a8ae-49e62f1db976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:23.977993 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff29873-d351-4bab-9d48-702eaa3e1a0b admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:23.984056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97dfd64-44e8-4c7e-82f9-c48556a0a331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:23.984777 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97dfd64-44e8-4c7e-82f9-c48556a0a331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:23.985070 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97dfd64-44e8-4c7e-82f9-c48556a0a331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:23.985556 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97dfd64-44e8-4c7e-82f9-c48556a0a331 admin admin] PATH_INFO: `/v3/users/eb4c352e955c4b60a580428ff5d8f824` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:23.986787 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97dfd64-44e8-4c7e-82f9-c48556a0a331 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb4c352e955c4b60a580428ff5d8f824)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:23.989056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97dfd64-44e8-4c7e-82f9-c48556a0a331 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:24.001233 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:24.008242 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-abc49f0c-d280-4c73-a8ae-49e62f1db976 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:24.010560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-abc49f0c-d280-4c73-a8ae-49e62f1db976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:24.026379 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1078/2153] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:24.031679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc49f0c-d280-4c73-a8ae-49e62f1db976 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:24.046283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc49f0c-d280-4c73-a8ae-49e62f1db976 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:24.055706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d613de0-b824-419a-8902-f156231cd3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:24.056359 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d613de0-b824-419a-8902-f156231cd3a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:24.056643 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d613de0-b824-419a-8902-f156231cd3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:24.056919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d613de0-b824-419a-8902-f156231cd3a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:24.057589 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d613de0-b824-419a-8902-f156231cd3a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:14:24.057930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d613de0-b824-419a-8902-f156231cd3a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:24.059798 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d613de0-b824-419a-8902-f156231cd3a6 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:24.090558 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1079/2154] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:14:24.092420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d613de0-b824-419a-8902-f156231cd3a6 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:24.105195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d97dfd64-44e8-4c7e-82f9-c48556a0a331 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb4c352e955c4b60a580428ff5d8f824'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:24.114335 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d613de0-b824-419a-8902-f156231cd3a6 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:24.119005 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d97dfd64-44e8-4c7e-82f9-c48556a0a331 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb4c352e955c4b60a580428ff5d8f824'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:24.127568 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc94ab7f-e8a8-472f-ae40-e233d2f8643c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:24.128230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94ab7f-e8a8-472f-ae40-e233d2f8643c admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:24.128505 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94ab7f-e8a8-472f-ae40-e233d2f8643c admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:24.128772 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94ab7f-e8a8-472f-ae40-e233d2f8643c admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:24.129546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc94ab7f-e8a8-472f-ae40-e233d2f8643c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:24.131600 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc94ab7f-e8a8-472f-ae40-e233d2f8643c admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:24.177968 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d97dfd64-44e8-4c7e-82f9-c48556a0a331 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb4c352e955c4b60a580428ff5d8f824'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:24.195531 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d97dfd64-44e8-4c7e-82f9-c48556a0a331 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb4c352e955c4b60a580428ff5d8f824'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:24.196883 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d97dfd64-44e8-4c7e-82f9-c48556a0a331 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb4c352e955c4b60a580428ff5d8f824'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:24.209770 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1080/2155] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:14:24 2022] POST /identity/v3/projects => generated 397 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:14:24.211388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc94ab7f-e8a8-472f-ae40-e233d2f8643c admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:24.213439 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-fc94ab7f-e8a8-472f-ae40-e233d2f8643c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:24.215083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1076/2156] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:14:23 2022] DELETE /identity/v3/users/eb4c352e955c4b60a580428ff5d8f824 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:24.328055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc94ab7f-e8a8-472f-ae40-e233d2f8643c admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:24.409621 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3a3b5-19e4-4cf0-89bd-6b5a7325d97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:24.410834 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3a3b5-19e4-4cf0-89bd-6b5a7325d97a admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:24.411640 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3a3b5-19e4-4cf0-89bd-6b5a7325d97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:24.412171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3a3b5-19e4-4cf0-89bd-6b5a7325d97a admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:24.413379 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3a3b5-19e4-4cf0-89bd-6b5a7325d97a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:24.417969 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3a3b5-19e4-4cf0-89bd-6b5a7325d97a admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:24.471214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1081/2157] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:14:24 2022] POST /identity/v3/users => generated 299 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:14:24.471825 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97dfd64-44e8-4c7e-82f9-c48556a0a331 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:24.492970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97dfd64-44e8-4c7e-82f9-c48556a0a331 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:24.500532 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8d402-30bf-46b7-86d7-e49c3f618da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:24.501446 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8d402-30bf-46b7-86d7-e49c3f618da3 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:24.501936 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8d402-30bf-46b7-86d7-e49c3f618da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:24.502372 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8d402-30bf-46b7-86d7-e49c3f618da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:24.503647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade8d402-30bf-46b7-86d7-e49c3f618da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:24.507326 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade8d402-30bf-46b7-86d7-e49c3f618da3 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:24.538907 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1077/2158] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:14:24.542680 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3a3b5-19e4-4cf0-89bd-6b5a7325d97a admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:24.558929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3a3b5-19e4-4cf0-89bd-6b5a7325d97a admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:24.572872 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68965b94-8404-44e5-9bc6-c742a6d6a201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:24.573512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68965b94-8404-44e5-9bc6-c742a6d6a201 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:24.573816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68965b94-8404-44e5-9bc6-c742a6d6a201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:24.574087 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68965b94-8404-44e5-9bc6-c742a6d6a201 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:24.574734 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68965b94-8404-44e5-9bc6-c742a6d6a201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:24.576915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68965b94-8404-44e5-9bc6-c742a6d6a201 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:24.606683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1082/2159] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:14:24.611864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8d402-30bf-46b7-86d7-e49c3f618da3 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:24.625616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8d402-30bf-46b7-86d7-e49c3f618da3 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:24.640020 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53267b5d-9811-4e76-aa06-a5cdf9fd0749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:24.640827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53267b5d-9811-4e76-aa06-a5cdf9fd0749 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:24.641028 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53267b5d-9811-4e76-aa06-a5cdf9fd0749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:24.641286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53267b5d-9811-4e76-aa06-a5cdf9fd0749 admin admin] PATH_INFO: `/v3/projects/d3f5fa68cae34a3ca767fa20993fb52d/users/0054be3ab7ef448aa5d21589023e33e5/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:24.660572 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53267b5d-9811-4e76-aa06-a5cdf9fd0749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3f5fa68cae34a3ca767fa20993fb52d, user_id=0054be3ab7ef448aa5d21589023e33e5, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:24.663383 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53267b5d-9811-4e76-aa06-a5cdf9fd0749 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:24.680941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1078/2160] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:14:24 2022] PUT /identity/v3/projects/d3f5fa68cae34a3ca767fa20993fb52d/users/0054be3ab7ef448aa5d21589023e33e5/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:24.694379 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db00e30-d2ea-42aa-8fc7-2f0f2e94fa5c None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:24.694663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db00e30-d2ea-42aa-8fc7-2f0f2e94fa5c None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:24.694938 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db00e30-d2ea-42aa-8fc7-2f0f2e94fa5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:24.747421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53267b5d-9811-4e76-aa06-a5cdf9fd0749 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:24.750342 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-53267b5d-9811-4e76-aa06-a5cdf9fd0749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:24.755044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-9db00e30-d2ea-42aa-8fc7-2f0f2e94fa5c None None] MFA Rules not processed for user `0054be3ab7ef448aa5d21589023e33e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:24.759034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-9db00e30-d2ea-42aa-8fc7-2f0f2e94fa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:24.790013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:24.824689 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:24.867335 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1083/2161] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:24 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:14:24.876252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db00e30-d2ea-42aa-8fc7-2f0f2e94fa5c None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:24.877924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-9db00e30-d2ea-42aa-8fc7-2f0f2e94fa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:24.890415 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53267b5d-9811-4e76-aa06-a5cdf9fd0749 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:24.906753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450372ed-7502-4f52-a2a5-31094bd522d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:24.908077 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450372ed-7502-4f52-a2a5-31094bd522d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:24.909780 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450372ed-7502-4f52-a2a5-31094bd522d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:24.910072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450372ed-7502-4f52-a2a5-31094bd522d6 admin admin] PATH_INFO: `/v3/projects/69f3b458bf5549aba6d9a92cf937157d` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:24.911433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450372ed-7502-4f52-a2a5-31094bd522d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69f3b458bf5549aba6d9a92cf937157d)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:24.913945 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450372ed-7502-4f52-a2a5-31094bd522d6 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:24.977257 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db00e30-d2ea-42aa-8fc7-2f0f2e94fa5c None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:24.994812 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-450372ed-7502-4f52-a2a5-31094bd522d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69f3b458bf5549aba6d9a92cf937157d', 'request_id': 'req-450372ed-7502-4f52-a2a5-31094bd522d6'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:25.004168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-450372ed-7502-4f52-a2a5-31094bd522d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69f3b458bf5549aba6d9a92cf937157d', 'request_id': 'req-450372ed-7502-4f52-a2a5-31094bd522d6'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:25.011870 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-450372ed-7502-4f52-a2a5-31094bd522d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69f3b458bf5549aba6d9a92cf937157d', 'request_id': 'req-450372ed-7502-4f52-a2a5-31094bd522d6'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:25.036068 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-450372ed-7502-4f52-a2a5-31094bd522d6 admin admin] The token cache is being invalidate because project 69f3b458bf5549aba6d9a92cf937157d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:14:25.036704 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-450372ed-7502-4f52-a2a5-31094bd522d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:25.038215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-450372ed-7502-4f52-a2a5-31094bd522d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:25.040764 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1079/2162] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:14:24 2022] DELETE /identity/v3/projects/69f3b458bf5549aba6d9a92cf937157d => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:25.046946 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60502334-e4c8-44fa-9e45-2b58cb7408b5 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:25.047571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60502334-e4c8-44fa-9e45-2b58cb7408b5 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:25.048193 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60502334-e4c8-44fa-9e45-2b58cb7408b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:25.052678 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d3e9e5-108e-4bc9-a4f8-a7ef3c056019 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:25.053458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3e9e5-108e-4bc9-a4f8-a7ef3c056019 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:25.053775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3e9e5-108e-4bc9-a4f8-a7ef3c056019 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:25.054056 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3e9e5-108e-4bc9-a4f8-a7ef3c056019 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:25.057078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-b4d3e9e5-108e-4bc9-a4f8-a7ef3c056019 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:25.078359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-60502334-e4c8-44fa-9e45-2b58cb7408b5 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:25.081920 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-60502334-e4c8-44fa-9e45-2b58cb7408b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:25.183102 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1080/2163] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:14:25.191604 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2cdf96-923f-4a31-a6c1-38f59910152c None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:25.191892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2cdf96-923f-4a31-a6c1-38f59910152c None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:25.192171 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2cdf96-923f-4a31-a6c1-38f59910152c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:25.199036 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d3e9e5-108e-4bc9-a4f8-a7ef3c056019 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:25.200714 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d3e9e5-108e-4bc9-a4f8-a7ef3c056019 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:25.226730 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1084/2164] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:24 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:14:25.230139 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-2f2cdf96-923f-4a31-a6c1-38f59910152c None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:25.231389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-2f2cdf96-923f-4a31-a6c1-38f59910152c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:25.252732 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1081/2165] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:14:25.260931 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d3e9e5-108e-4bc9-a4f8-a7ef3c056019 service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:25.274894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2cdf96-923f-4a31-a6c1-38f59910152c None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:25.276507 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d3e9e5-108e-4bc9-a4f8-a7ef3c056019 service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:25.276992 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-2f2cdf96-923f-4a31-a6c1-38f59910152c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:25.283728 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506bbf7-d808-482c-9af1-8ab79a280629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:25.284540 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506bbf7-d808-482c-9af1-8ab79a280629 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:25.285687 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506bbf7-d808-482c-9af1-8ab79a280629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:25.286611 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506bbf7-d808-482c-9af1-8ab79a280629 admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:25.288039 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506bbf7-d808-482c-9af1-8ab79a280629 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:14:25.288366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506bbf7-d808-482c-9af1-8ab79a280629 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:25.298813 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2cdf96-923f-4a31-a6c1-38f59910152c None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:25.315915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506bbf7-d808-482c-9af1-8ab79a280629 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:25.319785 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0973221-8b1a-4599-8146-f43e054d6c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:25.320599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0973221-8b1a-4599-8146-f43e054d6c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:25.320973 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0973221-8b1a-4599-8146-f43e054d6c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:25.321378 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0973221-8b1a-4599-8146-f43e054d6c68 admin admin] PATH_INFO: `/v3/users/2420a84d76e845c68bd14852b1a910a9` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:25.322918 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0973221-8b1a-4599-8146-f43e054d6c68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2420a84d76e845c68bd14852b1a910a9)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:25.325690 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0973221-8b1a-4599-8146-f43e054d6c68 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:25.354548 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1085/2166] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:14:25.363226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506bbf7-d808-482c-9af1-8ab79a280629 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:25.382572 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506bbf7-d808-482c-9af1-8ab79a280629 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:25.393047 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f98476-3404-48ef-94f1-5e522f400bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:25.394115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f98476-3404-48ef-94f1-5e522f400bbe admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:25.394609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f98476-3404-48ef-94f1-5e522f400bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:25.395043 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f98476-3404-48ef-94f1-5e522f400bbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:25.395921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f98476-3404-48ef-94f1-5e522f400bbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:25.399210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f98476-3404-48ef-94f1-5e522f400bbe admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:25.414291 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:25.452172 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a0973221-8b1a-4599-8146-f43e054d6c68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2420a84d76e845c68bd14852b1a910a9'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:25.465900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1086/2167] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:14:25 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:14:25.469248 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f98476-3404-48ef-94f1-5e522f400bbe admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:25.470289 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a0973221-8b1a-4599-8146-f43e054d6c68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2420a84d76e845c68bd14852b1a910a9'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:25.509000 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a0973221-8b1a-4599-8146-f43e054d6c68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2420a84d76e845c68bd14852b1a910a9'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:25.525665 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:25.526727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a0973221-8b1a-4599-8146-f43e054d6c68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2420a84d76e845c68bd14852b1a910a9'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:25.527802 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a0973221-8b1a-4599-8146-f43e054d6c68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2420a84d76e845c68bd14852b1a910a9'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:25.544283 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1082/2168] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:14:25 2022] DELETE /identity/v3/users/2420a84d76e845c68bd14852b1a910a9 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:25.582670 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f98476-3404-48ef-94f1-5e522f400bbe admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:25.708825 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d04993c-7cbb-4337-b67e-acc15697040b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:25.709522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04993c-7cbb-4337-b67e-acc15697040b admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:25.709914 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04993c-7cbb-4337-b67e-acc15697040b admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:25.710256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04993c-7cbb-4337-b67e-acc15697040b admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:25.710974 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d04993c-7cbb-4337-b67e-acc15697040b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:25.713281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d04993c-7cbb-4337-b67e-acc15697040b admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:25.744302 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1087/2169] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:14:25 2022] POST /identity/v3/users => generated 287 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:14:25.750477 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0973221-8b1a-4599-8146-f43e054d6c68 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:25.754439 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-a0973221-8b1a-4599-8146-f43e054d6c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:25.780105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0973221-8b1a-4599-8146-f43e054d6c68 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:25.794493 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c1fed0-da94-49dc-8580-d8ea2684a847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:25.797296 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1fed0-da94-49dc-8580-d8ea2684a847 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:25.797994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1fed0-da94-49dc-8580-d8ea2684a847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:25.798501 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1fed0-da94-49dc-8580-d8ea2684a847 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:25.799742 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c1fed0-da94-49dc-8580-d8ea2684a847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:25.802889 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c1fed0-da94-49dc-8580-d8ea2684a847 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:25.828282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1083/2170] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:25.833048 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d04993c-7cbb-4337-b67e-acc15697040b admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:25.849318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c1fed0-da94-49dc-8580-d8ea2684a847 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:25.850609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d04993c-7cbb-4337-b67e-acc15697040b admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:25.852149 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-f6c1fed0-da94-49dc-8580-d8ea2684a847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:25.856488 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acfa17-970c-472f-8625-d85893b425a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:25.858148 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acfa17-970c-472f-8625-d85893b425a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:25.858624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acfa17-970c-472f-8625-d85893b425a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:25.859089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acfa17-970c-472f-8625-d85893b425a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:25.860639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acfa17-970c-472f-8625-d85893b425a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:25.864656 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acfa17-970c-472f-8625-d85893b425a8 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:25.881758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1088/2171] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:25.886204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acfa17-970c-472f-8625-d85893b425a8 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:25.891231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c1fed0-da94-49dc-8580-d8ea2684a847 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:25.900035 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acfa17-970c-472f-8625-d85893b425a8 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:25.902645 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d09b8-d92d-49aa-a255-fafa5d330f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:25.903525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d09b8-d92d-49aa-a255-fafa5d330f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:25.904140 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d09b8-d92d-49aa-a255-fafa5d330f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:25.905357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d09b8-d92d-49aa-a255-fafa5d330f52 admin admin] PATH_INFO: `/v3/projects/bcff03cfe90e4d1bb5f9bf1564edafa6` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:25.907424 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76bbae6-0ec2-4e8d-807d-67767956fef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:25.907760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d09b8-d92d-49aa-a255-fafa5d330f52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcff03cfe90e4d1bb5f9bf1564edafa6)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:25.908108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76bbae6-0ec2-4e8d-807d-67767956fef6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:25.908403 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76bbae6-0ec2-4e8d-807d-67767956fef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:25.908722 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76bbae6-0ec2-4e8d-807d-67767956fef6 admin admin] PATH_INFO: `/v3/projects/95b900960ce84ebba14e9157e030287e/users/be17fa7c5ebe49efa4bcae7c721a5e01/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:25.914252 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d09b8-d92d-49aa-a255-fafa5d330f52 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:25.931546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76bbae6-0ec2-4e8d-807d-67767956fef6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95b900960ce84ebba14e9157e030287e, user_id=be17fa7c5ebe49efa4bcae7c721a5e01, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:25.934278 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76bbae6-0ec2-4e8d-807d-67767956fef6 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:25.962397 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1089/2172] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:14:25 2022] PUT /identity/v3/projects/95b900960ce84ebba14e9157e030287e/users/be17fa7c5ebe49efa4bcae7c721a5e01/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:14:25.968804 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810a4f3-fd40-461e-8003-32c1487f87dd None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:25.969404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810a4f3-fd40-461e-8003-32c1487f87dd None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:25.970039 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810a4f3-fd40-461e-8003-32c1487f87dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:26.001262 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-a810a4f3-fd40-461e-8003-32c1487f87dd None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:26.003701 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a810a4f3-fd40-461e-8003-32c1487f87dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:26.027766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-272d09b8-d92d-49aa-a255-fafa5d330f52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcff03cfe90e4d1bb5f9bf1564edafa6', 'request_id': 'req-272d09b8-d92d-49aa-a255-fafa5d330f52'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:26.039558 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-272d09b8-d92d-49aa-a255-fafa5d330f52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcff03cfe90e4d1bb5f9bf1564edafa6', 'request_id': 'req-272d09b8-d92d-49aa-a255-fafa5d330f52'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:26.056892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-272d09b8-d92d-49aa-a255-fafa5d330f52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcff03cfe90e4d1bb5f9bf1564edafa6', 'request_id': 'req-272d09b8-d92d-49aa-a255-fafa5d330f52'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:26.075428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-272d09b8-d92d-49aa-a255-fafa5d330f52 admin admin] The token cache is being invalidate because project bcff03cfe90e4d1bb5f9bf1564edafa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:14:26.075964 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-272d09b8-d92d-49aa-a255-fafa5d330f52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:26.077556 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-272d09b8-d92d-49aa-a255-fafa5d330f52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:26.079923 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1084/2173] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:14:25 2022] DELETE /identity/v3/projects/bcff03cfe90e4d1bb5f9bf1564edafa6 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:26.087493 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019541a-ab2e-4d56-ac50-5b3b6e72441d None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:26.088130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019541a-ab2e-4d56-ac50-5b3b6e72441d None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:26.088590 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019541a-ab2e-4d56-ac50-5b3b6e72441d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:26.125640 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1090/2174] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:26.130944 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-4019541a-ab2e-4d56-ac50-5b3b6e72441d None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:26.133267 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-4019541a-ab2e-4d56-ac50-5b3b6e72441d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:26.139455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810a4f3-fd40-461e-8003-32c1487f87dd None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:26.141062 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a810a4f3-fd40-461e-8003-32c1487f87dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:26.149367 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1085/2175] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:14:26.160105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fcb0b-817b-46fd-9433-58e697adc540 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:26.160857 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fcb0b-817b-46fd-9433-58e697adc540 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:26.161382 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fcb0b-817b-46fd-9433-58e697adc540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:26.171582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:26.207288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-746fcb0b-817b-46fd-9433-58e697adc540 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:26.208761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-746fcb0b-817b-46fd-9433-58e697adc540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:26.220142 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1086/2176] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:14:26.226372 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fcb0b-817b-46fd-9433-58e697adc540 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:26.242682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810a4f3-fd40-461e-8003-32c1487f87dd None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:26.243151 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fcb0b-817b-46fd-9433-58e697adc540 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:26.251719 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec5f42-3c75-4c65-afcb-a350ff9878cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:26.252694 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec5f42-3c75-4c65-afcb-a350ff9878cf service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:26.252880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642aa88-f7ce-42f8-946c-7cd11a6e35da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:26.253307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec5f42-3c75-4c65-afcb-a350ff9878cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:26.253758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec5f42-3c75-4c65-afcb-a350ff9878cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:26.253917 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642aa88-f7ce-42f8-946c-7cd11a6e35da admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:26.254267 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642aa88-f7ce-42f8-946c-7cd11a6e35da admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:26.254754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642aa88-f7ce-42f8-946c-7cd11a6e35da admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:26.256144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642aa88-f7ce-42f8-946c-7cd11a6e35da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:14:26.256803 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642aa88-f7ce-42f8-946c-7cd11a6e35da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:26.260138 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642aa88-f7ce-42f8-946c-7cd11a6e35da admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:26.270674 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec5f42-3c75-4c65-afcb-a350ff9878cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:26.273127 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec5f42-3c75-4c65-afcb-a350ff9878cf service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:26.289916 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1087/2177] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:14:26.296609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1091/2178] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:14:26.297113 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642aa88-f7ce-42f8-946c-7cd11a6e35da admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:26.315859 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642aa88-f7ce-42f8-946c-7cd11a6e35da admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:26.332815 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbdd38-e1cd-40ec-8f35-7ea090c76a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:26.336167 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbdd38-e1cd-40ec-8f35-7ea090c76a0b admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:26.336479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbdd38-e1cd-40ec-8f35-7ea090c76a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:26.336935 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbdd38-e1cd-40ec-8f35-7ea090c76a0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:26.338406 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fbdd38-e1cd-40ec-8f35-7ea090c76a0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:26.339981 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fbdd38-e1cd-40ec-8f35-7ea090c76a0b admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:26.402447 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1088/2179] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:14:26 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:14:26.407884 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec5f42-3c75-4c65-afcb-a350ff9878cf service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:26.428501 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec5f42-3c75-4c65-afcb-a350ff9878cf service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:26.595193 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda57bad-d2de-488a-b858-392c0e096e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:26.596140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda57bad-d2de-488a-b858-392c0e096e37 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:26.596140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda57bad-d2de-488a-b858-392c0e096e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:26.596140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda57bad-d2de-488a-b858-392c0e096e37 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:26.597008 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda57bad-d2de-488a-b858-392c0e096e37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:26.599187 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda57bad-d2de-488a-b858-392c0e096e37 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:26.639777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1092/2180] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:14:26 2022] POST /identity/v3/users => generated 294 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:14:26.641177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbdd38-e1cd-40ec-8f35-7ea090c76a0b admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:26.653910 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbdd38-e1cd-40ec-8f35-7ea090c76a0b admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:26.695968 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b172548-ba26-47ed-afc0-5fcaf02d96ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:26.697411 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b172548-ba26-47ed-afc0-5fcaf02d96ba admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:26.697949 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b172548-ba26-47ed-afc0-5fcaf02d96ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:26.698433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b172548-ba26-47ed-afc0-5fcaf02d96ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:26.708279 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b172548-ba26-47ed-afc0-5fcaf02d96ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:26.710573 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b172548-ba26-47ed-afc0-5fcaf02d96ba admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:26.728398 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1089/2181] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:26.735421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda57bad-d2de-488a-b858-392c0e096e37 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:26.762648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda57bad-d2de-488a-b858-392c0e096e37 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:26.775025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e758f7-aefb-458a-a684-c71966865ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:26.776691 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e758f7-aefb-458a-a684-c71966865ebf admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:26.778091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e758f7-aefb-458a-a684-c71966865ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:26.778444 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e758f7-aefb-458a-a684-c71966865ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:26.780790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e758f7-aefb-458a-a684-c71966865ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:26.783737 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e758f7-aefb-458a-a684-c71966865ebf admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:26.824626 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1093/2182] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:26.832445 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b172548-ba26-47ed-afc0-5fcaf02d96ba admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:26.853048 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b172548-ba26-47ed-afc0-5fcaf02d96ba admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:26.859697 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f5d25-d2e4-45db-b5a3-20a9dde9ef39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:26.860381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f5d25-d2e4-45db-b5a3-20a9dde9ef39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:26.861137 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f5d25-d2e4-45db-b5a3-20a9dde9ef39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:26.861137 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f5d25-d2e4-45db-b5a3-20a9dde9ef39 admin admin] PATH_INFO: `/v3/projects/fc733dcfcf9043feb354598fc2792ff5/users/1ffc617c50984f83adc3269b118c8a91/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:26.895426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f5d25-d2e4-45db-b5a3-20a9dde9ef39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc733dcfcf9043feb354598fc2792ff5, user_id=1ffc617c50984f83adc3269b118c8a91, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:26.901484 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f5d25-d2e4-45db-b5a3-20a9dde9ef39 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:26.919265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:26.928862 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1090/2183] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:14:26 2022] PUT /identity/v3/projects/fc733dcfcf9043feb354598fc2792ff5/users/1ffc617c50984f83adc3269b118c8a91/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:26.936745 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d44421-fcde-40d9-8cdd-7467d15af92f None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:26.937124 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d44421-fcde-40d9-8cdd-7467d15af92f None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:26.937498 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d44421-fcde-40d9-8cdd-7467d15af92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:26.976155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-56d44421-fcde-40d9-8cdd-7467d15af92f None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:26.978494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-56d44421-fcde-40d9-8cdd-7467d15af92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:27.148141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1094/2184] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:14:27.164635 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f5d25-d2e4-45db-b5a3-20a9dde9ef39 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:27.168112 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-251f5d25-d2e4-45db-b5a3-20a9dde9ef39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:27.324597 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f5d25-d2e4-45db-b5a3-20a9dde9ef39 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:27.330655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889e346-6c5a-4059-9fe6-5e40ae203ac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:27.331466 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889e346-6c5a-4059-9fe6-5e40ae203ac7 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:27.331665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889e346-6c5a-4059-9fe6-5e40ae203ac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:27.331940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889e346-6c5a-4059-9fe6-5e40ae203ac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:27.348440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889e346-6c5a-4059-9fe6-5e40ae203ac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:27.350063 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889e346-6c5a-4059-9fe6-5e40ae203ac7 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:27.369186 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1091/2185] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:14:27.705452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc0d7b-3333-4b98-a0d0-10f2a5335a3f None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:27.706058 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc0d7b-3333-4b98-a0d0-10f2a5335a3f None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:27.706058 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc0d7b-3333-4b98-a0d0-10f2a5335a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:27.707152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889e346-6c5a-4059-9fe6-5e40ae203ac7 service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:27.708614 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-6889e346-6c5a-4059-9fe6-5e40ae203ac7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:27.734077 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889e346-6c5a-4059-9fe6-5e40ae203ac7 service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:27.745978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c006503-6a4f-4819-81e1-c04440bf44a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:27.747061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c006503-6a4f-4819-81e1-c04440bf44a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:27.747664 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c006503-6a4f-4819-81e1-c04440bf44a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:27.748207 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c006503-6a4f-4819-81e1-c04440bf44a0 admin admin] PATH_INFO: `/v3/users/309ca8171b0345f59697148cc62c2409` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:27.750295 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c006503-6a4f-4819-81e1-c04440bf44a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=309ca8171b0345f59697148cc62c2409)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:27.753751 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c006503-6a4f-4819-81e1-c04440bf44a0 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:27.774962 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-abfc0d7b-3333-4b98-a0d0-10f2a5335a3f None None] MFA Rules not processed for user `be17fa7c5ebe49efa4bcae7c721a5e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:27.776247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-abfc0d7b-3333-4b98-a0d0-10f2a5335a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:27.890545 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-5c006503-6a4f-4819-81e1-c04440bf44a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '309ca8171b0345f59697148cc62c2409'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:27.900555 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:27.910277 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-5c006503-6a4f-4819-81e1-c04440bf44a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '309ca8171b0345f59697148cc62c2409'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:27.963361 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-5c006503-6a4f-4819-81e1-c04440bf44a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '309ca8171b0345f59697148cc62c2409'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:27.995717 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-5c006503-6a4f-4819-81e1-c04440bf44a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '309ca8171b0345f59697148cc62c2409'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:28.000662 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-5c006503-6a4f-4819-81e1-c04440bf44a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '309ca8171b0345f59697148cc62c2409'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:28.013063 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1095/2186] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:28.035998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc0d7b-3333-4b98-a0d0-10f2a5335a3f None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:28.038461 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-abfc0d7b-3333-4b98-a0d0-10f2a5335a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:28.047346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1092/2187] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:14:27 2022] DELETE /identity/v3/users/309ca8171b0345f59697148cc62c2409 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:28.054323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75512310-bd01-4d66-9ece-744d49968082 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:28.054794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75512310-bd01-4d66-9ece-744d49968082 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:28.055365 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75512310-bd01-4d66-9ece-744d49968082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:28.119004 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-75512310-bd01-4d66-9ece-744d49968082 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:28.120338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-75512310-bd01-4d66-9ece-744d49968082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:28.187708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc0d7b-3333-4b98-a0d0-10f2a5335a3f None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:28.252898 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1093/2188] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:28.261548 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75512310-bd01-4d66-9ece-744d49968082 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:28.263571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-75512310-bd01-4d66-9ece-744d49968082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:28.307746 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:28.332514 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f83fc4-cbac-4b62-8926-3abda5be3da8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:28.333336 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f83fc4-cbac-4b62-8926-3abda5be3da8 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:28.333777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f83fc4-cbac-4b62-8926-3abda5be3da8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:28.334160 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f83fc4-cbac-4b62-8926-3abda5be3da8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:28.336329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c6f83fc4-cbac-4b62-8926-3abda5be3da8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:28.443959 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75512310-bd01-4d66-9ece-744d49968082 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:28.453202 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ada44-0018-4c8f-940b-e6e4be95dd7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:28.454203 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ada44-0018-4c8f-940b-e6e4be95dd7b service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:28.454704 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ada44-0018-4c8f-940b-e6e4be95dd7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:28.455396 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ada44-0018-4c8f-940b-e6e4be95dd7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:28.481377 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f83fc4-cbac-4b62-8926-3abda5be3da8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:28.483578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f83fc4-cbac-4b62-8926-3abda5be3da8 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:28.484930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ada44-0018-4c8f-940b-e6e4be95dd7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:28.488499 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ada44-0018-4c8f-940b-e6e4be95dd7b service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:28.512095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1096/2189] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:14:28.542716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1094/2190] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:14:29.004745 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f83fc4-cbac-4b62-8926-3abda5be3da8 service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:29.007622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c6f83fc4-cbac-4b62-8926-3abda5be3da8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:29.035153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f83fc4-cbac-4b62-8926-3abda5be3da8 service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:29.043871 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4343c8-bad3-49ec-8dfb-120bcb0e63f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:29.045440 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4343c8-bad3-49ec-8dfb-120bcb0e63f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:29.046070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4343c8-bad3-49ec-8dfb-120bcb0e63f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:29.046465 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4343c8-bad3-49ec-8dfb-120bcb0e63f6 admin admin] PATH_INFO: `/v3/projects/79a173ba61d54c9eaf06e0c960ac1249` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:29.051241 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4343c8-bad3-49ec-8dfb-120bcb0e63f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79a173ba61d54c9eaf06e0c960ac1249)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:29.053666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4343c8-bad3-49ec-8dfb-120bcb0e63f6 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:29.102652 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:29.115740 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8a4343c8-bad3-49ec-8dfb-120bcb0e63f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79a173ba61d54c9eaf06e0c960ac1249', 'request_id': 'req-8a4343c8-bad3-49ec-8dfb-120bcb0e63f6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:29.121757 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8a4343c8-bad3-49ec-8dfb-120bcb0e63f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79a173ba61d54c9eaf06e0c960ac1249', 'request_id': 'req-8a4343c8-bad3-49ec-8dfb-120bcb0e63f6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:29.132531 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8a4343c8-bad3-49ec-8dfb-120bcb0e63f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79a173ba61d54c9eaf06e0c960ac1249', 'request_id': 'req-8a4343c8-bad3-49ec-8dfb-120bcb0e63f6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:29.148085 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8a4343c8-bad3-49ec-8dfb-120bcb0e63f6 admin admin] The token cache is being invalidate because project 79a173ba61d54c9eaf06e0c960ac1249 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:14:29.148546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8a4343c8-bad3-49ec-8dfb-120bcb0e63f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:29.150286 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8a4343c8-bad3-49ec-8dfb-120bcb0e63f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:29.152527 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1097/2191] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:14:29 2022] DELETE /identity/v3/projects/79a173ba61d54c9eaf06e0c960ac1249 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:29.157975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630db80-0f03-4ad9-a6b1-91c5fd3908e5 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:29.158517 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630db80-0f03-4ad9-a6b1-91c5fd3908e5 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:29.158517 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630db80-0f03-4ad9-a6b1-91c5fd3908e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:29.180769 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-0630db80-0f03-4ad9-a6b1-91c5fd3908e5 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:29.182578 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-0630db80-0f03-4ad9-a6b1-91c5fd3908e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:29.268343 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1095/2192] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:29.277249 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996dd2eb-eaaa-47cc-9669-425ab46f7a24 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:29.277249 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996dd2eb-eaaa-47cc-9669-425ab46f7a24 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:29.277734 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996dd2eb-eaaa-47cc-9669-425ab46f7a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:29.300355 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-996dd2eb-eaaa-47cc-9669-425ab46f7a24 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:29.301851 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-996dd2eb-eaaa-47cc-9669-425ab46f7a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:29.311788 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1098/2193] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:14:29.316153 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630db80-0f03-4ad9-a6b1-91c5fd3908e5 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:29.327746 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630db80-0f03-4ad9-a6b1-91c5fd3908e5 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:29.335469 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc6b16d-0f49-481b-95cb-c51a3f4b4da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:29.336380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6b16d-0f49-481b-95cb-c51a3f4b4da2 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:29.336380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6b16d-0f49-481b-95cb-c51a3f4b4da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:29.336815 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6b16d-0f49-481b-95cb-c51a3f4b4da2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:29.337509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc6b16d-0f49-481b-95cb-c51a3f4b4da2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:14:29.337862 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc6b16d-0f49-481b-95cb-c51a3f4b4da2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:29.339523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc6b16d-0f49-481b-95cb-c51a3f4b4da2 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:29.356621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1096/2194] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:14:29.358677 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996dd2eb-eaaa-47cc-9669-425ab46f7a24 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:29.369155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996dd2eb-eaaa-47cc-9669-425ab46f7a24 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:29.375208 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea71944-3d1f-4403-96d2-c3f8cca97116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:29.375657 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea71944-3d1f-4403-96d2-c3f8cca97116 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:29.375930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea71944-3d1f-4403-96d2-c3f8cca97116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:29.376201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea71944-3d1f-4403-96d2-c3f8cca97116 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:29.376863 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea71944-3d1f-4403-96d2-c3f8cca97116 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:29.378882 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea71944-3d1f-4403-96d2-c3f8cca97116 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:29.413798 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1099/2195] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:14:29 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:14:29.415151 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc6b16d-0f49-481b-95cb-c51a3f4b4da2 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:29.427174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc6b16d-0f49-481b-95cb-c51a3f4b4da2 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:29.497360 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b85a10-a981-4851-951d-2eb24d293085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:29.498074 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b85a10-a981-4851-951d-2eb24d293085 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:29.498539 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b85a10-a981-4851-951d-2eb24d293085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:29.499102 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b85a10-a981-4851-951d-2eb24d293085 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:29.499826 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b85a10-a981-4851-951d-2eb24d293085 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:29.501944 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b85a10-a981-4851-951d-2eb24d293085 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:29.527309 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1097/2196] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:14:29 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:14:29.533090 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea71944-3d1f-4403-96d2-c3f8cca97116 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:29.546637 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea71944-3d1f-4403-96d2-c3f8cca97116 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:29.552479 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6552fb6-bbc5-4684-b82d-19d0c57b118a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:29.553421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6552fb6-bbc5-4684-b82d-19d0c57b118a admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:29.553800 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6552fb6-bbc5-4684-b82d-19d0c57b118a admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:29.554203 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6552fb6-bbc5-4684-b82d-19d0c57b118a admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:29.554940 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6552fb6-bbc5-4684-b82d-19d0c57b118a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:29.556914 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6552fb6-bbc5-4684-b82d-19d0c57b118a admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:29.570112 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1100/2197] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:29.575603 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b85a10-a981-4851-951d-2eb24d293085 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:29.588562 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b85a10-a981-4851-951d-2eb24d293085 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:29.596826 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f29dd5-cad4-40b3-9660-b725aa8fb284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:29.597529 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f29dd5-cad4-40b3-9660-b725aa8fb284 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:29.597823 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f29dd5-cad4-40b3-9660-b725aa8fb284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:29.598097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f29dd5-cad4-40b3-9660-b725aa8fb284 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:29.598750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f29dd5-cad4-40b3-9660-b725aa8fb284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:29.600362 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f29dd5-cad4-40b3-9660-b725aa8fb284 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:29.619777 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1098/2198] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:29.625245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6552fb6-bbc5-4684-b82d-19d0c57b118a admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:29.643995 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6552fb6-bbc5-4684-b82d-19d0c57b118a admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:29.650582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01c063-2a2b-408e-827a-55ef143c9480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:29.651275 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01c063-2a2b-408e-827a-55ef143c9480 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:29.651561 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01c063-2a2b-408e-827a-55ef143c9480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:29.651832 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01c063-2a2b-408e-827a-55ef143c9480 admin admin] PATH_INFO: `/v3/projects/89a2efd044d546e9b4648892a85e4b48/users/631741079603419581c44c0bcb560b7f/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:29.685121 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01c063-2a2b-408e-827a-55ef143c9480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89a2efd044d546e9b4648892a85e4b48, user_id=631741079603419581c44c0bcb560b7f, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:29.691393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01c063-2a2b-408e-827a-55ef143c9480 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:29.723562 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11b752-5db4-4a2d-b52a-23b29b6ed5d1 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:29.723897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11b752-5db4-4a2d-b52a-23b29b6ed5d1 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:29.724181 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11b752-5db4-4a2d-b52a-23b29b6ed5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:29.725071 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1101/2199] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:14:29 2022] PUT /identity/v3/projects/89a2efd044d546e9b4648892a85e4b48/users/631741079603419581c44c0bcb560b7f/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:29.769020 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-7b11b752-5db4-4a2d-b52a-23b29b6ed5d1 None None] MFA Rules not processed for user `631741079603419581c44c0bcb560b7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:29.770593 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-7b11b752-5db4-4a2d-b52a-23b29b6ed5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:29.796891 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:29.888397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1099/2200] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:14:29.894144 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01c063-2a2b-408e-827a-55ef143c9480 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:29.979981 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01c063-2a2b-408e-827a-55ef143c9480 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:29.990672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420e844-bff8-4801-bae0-90b3835e8100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:29.999975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420e844-bff8-4801-bae0-90b3835e8100 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:30.000733 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420e844-bff8-4801-bae0-90b3835e8100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:30.001050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420e844-bff8-4801-bae0-90b3835e8100 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:30.001905 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420e844-bff8-4801-bae0-90b3835e8100 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:30.005463 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420e844-bff8-4801-bae0-90b3835e8100 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:30.041978 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1102/2201] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:14:29 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:14:30.044458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11b752-5db4-4a2d-b52a-23b29b6ed5d1 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:30.057342 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11b752-5db4-4a2d-b52a-23b29b6ed5d1 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:30.144647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582432f-2173-46b4-b115-ac9cf49b91c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:30.145369 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582432f-2173-46b4-b115-ac9cf49b91c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:30.145930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582432f-2173-46b4-b115-ac9cf49b91c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:30.146034 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582432f-2173-46b4-b115-ac9cf49b91c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:30.146756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c582432f-2173-46b4-b115-ac9cf49b91c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:30.148778 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c582432f-2173-46b4-b115-ac9cf49b91c5 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:30.187298 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1100/2202] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:14:30 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:14:30.189381 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420e844-bff8-4801-bae0-90b3835e8100 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:30.208123 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420e844-bff8-4801-bae0-90b3835e8100 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:30.214735 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e30ccf-6b76-4db6-add0-e78b343c9185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:30.215490 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e30ccf-6b76-4db6-add0-e78b343c9185 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:30.215826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e30ccf-6b76-4db6-add0-e78b343c9185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:30.216163 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e30ccf-6b76-4db6-add0-e78b343c9185 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:30.217426 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e30ccf-6b76-4db6-add0-e78b343c9185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:30.219697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e30ccf-6b76-4db6-add0-e78b343c9185 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:30.242481 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1103/2203] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:30.246891 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582432f-2173-46b4-b115-ac9cf49b91c5 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:30.259906 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582432f-2173-46b4-b115-ac9cf49b91c5 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:30.265162 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad6d5cb-0737-46df-8f9a-e1a6450b80ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:30.265830 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6d5cb-0737-46df-8f9a-e1a6450b80ef admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:30.266114 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6d5cb-0737-46df-8f9a-e1a6450b80ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:30.266388 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6d5cb-0737-46df-8f9a-e1a6450b80ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:30.267111 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad6d5cb-0737-46df-8f9a-e1a6450b80ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:30.268691 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad6d5cb-0737-46df-8f9a-e1a6450b80ef admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:30.281996 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1101/2204] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:30.286189 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e30ccf-6b76-4db6-add0-e78b343c9185 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:30.300084 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e30ccf-6b76-4db6-add0-e78b343c9185 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:30.309176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ed4e1-2727-40f3-8f3b-e02d047c9604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:30.309176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ed4e1-2727-40f3-8f3b-e02d047c9604 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:30.309176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ed4e1-2727-40f3-8f3b-e02d047c9604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:30.309176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ed4e1-2727-40f3-8f3b-e02d047c9604 admin admin] PATH_INFO: `/v3/projects/8dfc94da805c47a2887f1aec130e7280/users/53b2cdb3b83f4265b650bac712838532/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:30.332738 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ed4e1-2727-40f3-8f3b-e02d047c9604 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dfc94da805c47a2887f1aec130e7280, user_id=53b2cdb3b83f4265b650bac712838532, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:30.335666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ed4e1-2727-40f3-8f3b-e02d047c9604 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:30.351561 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1104/2205] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:14:30 2022] PUT /identity/v3/projects/8dfc94da805c47a2887f1aec130e7280/users/53b2cdb3b83f4265b650bac712838532/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:30.363654 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d63f0-bb80-4915-87c8-b992ca44acda None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:30.363942 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d63f0-bb80-4915-87c8-b992ca44acda None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:30.364218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d63f0-bb80-4915-87c8-b992ca44acda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:30.395835 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-120d63f0-bb80-4915-87c8-b992ca44acda None None] MFA Rules not processed for user `53b2cdb3b83f4265b650bac712838532`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:30.397094 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-120d63f0-bb80-4915-87c8-b992ca44acda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:30.473868 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1102/2206] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:30.477452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ed4e1-2727-40f3-8f3b-e02d047c9604 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:30.489024 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:30.548392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ed4e1-2727-40f3-8f3b-e02d047c9604 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:30.553251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0152cee-87cd-4d86-aa1e-c1f3754a1223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:30.553919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0152cee-87cd-4d86-aa1e-c1f3754a1223 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:30.554202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0152cee-87cd-4d86-aa1e-c1f3754a1223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:30.554471 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0152cee-87cd-4d86-aa1e-c1f3754a1223 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:30.555114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0152cee-87cd-4d86-aa1e-c1f3754a1223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:30.557065 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0152cee-87cd-4d86-aa1e-c1f3754a1223 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:30.595560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1105/2207] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:14:30 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:14:30.596850 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d63f0-bb80-4915-87c8-b992ca44acda None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:30.609627 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d63f0-bb80-4915-87c8-b992ca44acda None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:30.650816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:30.758872 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0544d-f883-480d-aa09-3476c4c73d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:30.758872 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0544d-f883-480d-aa09-3476c4c73d59 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:30.758872 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0544d-f883-480d-aa09-3476c4c73d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:30.758872 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0544d-f883-480d-aa09-3476c4c73d59 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:30.759493 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0544d-f883-480d-aa09-3476c4c73d59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:30.764904 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0544d-f883-480d-aa09-3476c4c73d59 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:30.802788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1103/2208] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:14:30 2022] POST /identity/v3/users => generated 294 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:14:30.806443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0152cee-87cd-4d86-aa1e-c1f3754a1223 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:30.831732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0152cee-87cd-4d86-aa1e-c1f3754a1223 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:30.840057 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f14146-875d-4308-8d8a-6b70da76dcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:30.840694 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f14146-875d-4308-8d8a-6b70da76dcd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:30.840980 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f14146-875d-4308-8d8a-6b70da76dcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:30.841282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f14146-875d-4308-8d8a-6b70da76dcd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:30.842647 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f14146-875d-4308-8d8a-6b70da76dcd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:30.844549 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f14146-875d-4308-8d8a-6b70da76dcd6 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:30.872888 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1106/2209] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:30.879286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0544d-f883-480d-aa09-3476c4c73d59 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:30.903431 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0544d-f883-480d-aa09-3476c4c73d59 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:30.918398 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120d7bc-8b52-4796-b472-955a632799a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:30.921357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120d7bc-8b52-4796-b472-955a632799a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:30.923894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120d7bc-8b52-4796-b472-955a632799a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:30.924469 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120d7bc-8b52-4796-b472-955a632799a6 admin admin] PATH_INFO: `/v3/projects/995f4ac780b047948405bc727ecbc240/users/6814d8f5f70242a4b797c4a1842471d0/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:30.972096 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2120d7bc-8b52-4796-b472-955a632799a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=995f4ac780b047948405bc727ecbc240, user_id=6814d8f5f70242a4b797c4a1842471d0, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:30.975142 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2120d7bc-8b52-4796-b472-955a632799a6 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:31.025245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1104/2210] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:14:30 2022] PUT /identity/v3/projects/995f4ac780b047948405bc727ecbc240/users/6814d8f5f70242a4b797c4a1842471d0/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:14:31.026752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f14146-875d-4308-8d8a-6b70da76dcd6 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:31.041898 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f14146-875d-4308-8d8a-6b70da76dcd6 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:31.158999 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a24c7-40d7-4603-8cfe-3a6622f320ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:31.159733 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a24c7-40d7-4603-8cfe-3a6622f320ad admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:31.160174 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a24c7-40d7-4603-8cfe-3a6622f320ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:31.160472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a24c7-40d7-4603-8cfe-3a6622f320ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:31.172754 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a24c7-40d7-4603-8cfe-3a6622f320ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:31.175552 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a24c7-40d7-4603-8cfe-3a6622f320ad admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:31.210829 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1107/2211] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:31.217988 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120d7bc-8b52-4796-b472-955a632799a6 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:31.233827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120d7bc-8b52-4796-b472-955a632799a6 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:31.255353 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6957c-1105-4db6-85a1-7797e3e5cba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:31.256272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6957c-1105-4db6-85a1-7797e3e5cba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:31.256705 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6957c-1105-4db6-85a1-7797e3e5cba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:31.257156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6957c-1105-4db6-85a1-7797e3e5cba7 admin admin] PATH_INFO: `/v3/projects/995f4ac780b047948405bc727ecbc240/users/6814d8f5f70242a4b797c4a1842471d0/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:31.259397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6957c-1105-4db6-85a1-7797e3e5cba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=995f4ac780b047948405bc727ecbc240, user_id=6814d8f5f70242a4b797c4a1842471d0, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:31.263333 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6957c-1105-4db6-85a1-7797e3e5cba7 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:31.283363 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1105/2212] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:14:31 2022] PUT /identity/v3/projects/995f4ac780b047948405bc727ecbc240/users/6814d8f5f70242a4b797c4a1842471d0/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:31.294225 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f81970-46ca-485a-b8d5-20999ea37c08 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:31.294589 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f81970-46ca-485a-b8d5-20999ea37c08 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:31.295641 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f81970-46ca-485a-b8d5-20999ea37c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:31.347676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-a7f81970-46ca-485a-b8d5-20999ea37c08 None None] MFA Rules not processed for user `6814d8f5f70242a4b797c4a1842471d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:31.348926 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a7f81970-46ca-485a-b8d5-20999ea37c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:31.450507 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1108/2213] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:31.468849 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6957c-1105-4db6-85a1-7797e3e5cba7 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:31.475379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-35c6957c-1105-4db6-85a1-7797e3e5cba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:31.584068 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6957c-1105-4db6-85a1-7797e3e5cba7 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:31.588803 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c2044-b08b-40cf-b529-cba279349978 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:31.589540 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c2044-b08b-40cf-b529-cba279349978 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:31.589917 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c2044-b08b-40cf-b529-cba279349978 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:31.590255 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c2044-b08b-40cf-b529-cba279349978 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:31.709246 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c2044-b08b-40cf-b529-cba279349978 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:31.713605 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c2044-b08b-40cf-b529-cba279349978 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:31.731473 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1106/2214] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:14:33.605412 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a85b1c-6344-431e-8f43-c9381510b206 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:33.605954 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a85b1c-6344-431e-8f43-c9381510b206 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:33.606351 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a85b1c-6344-431e-8f43-c9381510b206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:33.629613 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:33.670916 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-43a85b1c-6344-431e-8f43-c9381510b206 None None] MFA Rules not processed for user `1ffc617c50984f83adc3269b118c8a91`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:33.674373 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-43a85b1c-6344-431e-8f43-c9381510b206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:33.815763 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1109/2215] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:33 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:33.839517 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c2044-b08b-40cf-b529-cba279349978 service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:33.845582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-ab4c2044-b08b-40cf-b529-cba279349978 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:33.930118 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:34.021500 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c2044-b08b-40cf-b529-cba279349978 service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:34.030102 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb7c71-889f-48a3-92f8-630c5c9caedf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:34.031901 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb7c71-889f-48a3-92f8-630c5c9caedf service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:34.032295 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb7c71-889f-48a3-92f8-630c5c9caedf service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:34.032688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb7c71-889f-48a3-92f8-630c5c9caedf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:34.046847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb7c71-889f-48a3-92f8-630c5c9caedf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:34.049676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb7c71-889f-48a3-92f8-630c5c9caedf service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:34.064663 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1107/2216] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:33 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:14:35.668276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758b5f6-8947-477e-b3d6-c3d3d2ff3c01 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:35.671296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758b5f6-8947-477e-b3d6-c3d3d2ff3c01 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:35.671897 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758b5f6-8947-477e-b3d6-c3d3d2ff3c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:35.712773 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-0758b5f6-8947-477e-b3d6-c3d3d2ff3c01 None None] MFA Rules not processed for user `be17fa7c5ebe49efa4bcae7c721a5e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:35.714307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-0758b5f6-8947-477e-b3d6-c3d3d2ff3c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:35.842543 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1110/2217] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:35.911286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb7c71-889f-48a3-92f8-630c5c9caedf service nova] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:35.938047 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb7c71-889f-48a3-92f8-630c5c9caedf service nova] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:35.944930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735179e-ad6f-4943-9b98-765d5748fcbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:35.945839 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735179e-ad6f-4943-9b98-765d5748fcbd service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:35.946687 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735179e-ad6f-4943-9b98-765d5748fcbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:35.947258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735179e-ad6f-4943-9b98-765d5748fcbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:36.059755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735179e-ad6f-4943-9b98-765d5748fcbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:36.063144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735179e-ad6f-4943-9b98-765d5748fcbd service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:36.082492 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1108/2218] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:14:36.759508 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369d490-9ea6-4927-bd15-baa6afd07602 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:36.760042 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369d490-9ea6-4927-bd15-baa6afd07602 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:36.760523 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369d490-9ea6-4927-bd15-baa6afd07602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:36.806141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-7369d490-9ea6-4927-bd15-baa6afd07602 None None] MFA Rules not processed for user `1ffc617c50984f83adc3269b118c8a91`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:36.808234 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-7369d490-9ea6-4927-bd15-baa6afd07602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:36.829079 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1111/2219] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:37.438493 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735179e-ad6f-4943-9b98-765d5748fcbd service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:37.440704 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-b735179e-ad6f-4943-9b98-765d5748fcbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:37.493346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735179e-ad6f-4943-9b98-765d5748fcbd service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:37.528708 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260743b0-fd73-4b6d-a617-bd98e23b7b63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:37.529843 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260743b0-fd73-4b6d-a617-bd98e23b7b63 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:37.530219 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260743b0-fd73-4b6d-a617-bd98e23b7b63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:37.530871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260743b0-fd73-4b6d-a617-bd98e23b7b63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:37.604635 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260743b0-fd73-4b6d-a617-bd98e23b7b63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:37.618423 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260743b0-fd73-4b6d-a617-bd98e23b7b63 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:37.639911 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369d490-9ea6-4927-bd15-baa6afd07602 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:37.645303 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-7369d490-9ea6-4927-bd15-baa6afd07602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:37.667098 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1109/2220] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:37 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:14:37.960979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369d490-9ea6-4927-bd15-baa6afd07602 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:37.980196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ae418-cff8-4807-8d4d-03915534c5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:37.981272 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ae418-cff8-4807-8d4d-03915534c5a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:37.981790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ae418-cff8-4807-8d4d-03915534c5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:37.986619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ae418-cff8-4807-8d4d-03915534c5a7 admin admin] PATH_INFO: `/v3/users/0054be3ab7ef448aa5d21589023e33e5` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:37.991186 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ae418-cff8-4807-8d4d-03915534c5a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0054be3ab7ef448aa5d21589023e33e5)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:37.994161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ae418-cff8-4807-8d4d-03915534c5a7 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:38.091540 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:38.117115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-601ae418-cff8-4807-8d4d-03915534c5a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0054be3ab7ef448aa5d21589023e33e5'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:38.179247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-601ae418-cff8-4807-8d4d-03915534c5a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0054be3ab7ef448aa5d21589023e33e5'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:38.202085 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-601ae418-cff8-4807-8d4d-03915534c5a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0054be3ab7ef448aa5d21589023e33e5'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:38.225577 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-601ae418-cff8-4807-8d4d-03915534c5a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0054be3ab7ef448aa5d21589023e33e5'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:38.241420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-601ae418-cff8-4807-8d4d-03915534c5a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0054be3ab7ef448aa5d21589023e33e5'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:38.265732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1112/2221] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:14:37 2022] DELETE /identity/v3/users/0054be3ab7ef448aa5d21589023e33e5 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:14:38.272330 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4d47e-fef5-46f8-9c17-27ef2c0606b3 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:38.273309 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4d47e-fef5-46f8-9c17-27ef2c0606b3 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:38.274097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4d47e-fef5-46f8-9c17-27ef2c0606b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:38.345555 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-e1c4d47e-fef5-46f8-9c17-27ef2c0606b3 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:38.347956 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-e1c4d47e-fef5-46f8-9c17-27ef2c0606b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:38.517510 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1110/2222] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:38.529014 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ae418-cff8-4807-8d4d-03915534c5a7 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:38.531550 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-601ae418-cff8-4807-8d4d-03915534c5a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:38.668238 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ae418-cff8-4807-8d4d-03915534c5a7 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:38.681757 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324dd1e-826d-46d8-ae66-ff57adcf221b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:38.682464 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324dd1e-826d-46d8-ae66-ff57adcf221b service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:38.682992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324dd1e-826d-46d8-ae66-ff57adcf221b service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:38.684983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324dd1e-826d-46d8-ae66-ff57adcf221b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:38.723607 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324dd1e-826d-46d8-ae66-ff57adcf221b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:38.730112 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324dd1e-826d-46d8-ae66-ff57adcf221b service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:38.768872 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1113/2223] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:14:39.593211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4d47e-fef5-46f8-9c17-27ef2c0606b3 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:39.595630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-e1c4d47e-fef5-46f8-9c17-27ef2c0606b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:39.627984 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4d47e-fef5-46f8-9c17-27ef2c0606b3 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:39.658726 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8134e72f-80d2-4446-a618-a442b9ef9879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:39.661753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134e72f-80d2-4446-a618-a442b9ef9879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:39.664004 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134e72f-80d2-4446-a618-a442b9ef9879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:39.665331 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134e72f-80d2-4446-a618-a442b9ef9879 admin admin] PATH_INFO: `/v3/projects/d3f5fa68cae34a3ca767fa20993fb52d` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:39.667689 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8134e72f-80d2-4446-a618-a442b9ef9879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3f5fa68cae34a3ca767fa20993fb52d)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:39.673835 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8134e72f-80d2-4446-a618-a442b9ef9879 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:39.788814 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:39.812659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8134e72f-80d2-4446-a618-a442b9ef9879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3f5fa68cae34a3ca767fa20993fb52d', 'request_id': 'req-8134e72f-80d2-4446-a618-a442b9ef9879'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:39.820527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8134e72f-80d2-4446-a618-a442b9ef9879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3f5fa68cae34a3ca767fa20993fb52d', 'request_id': 'req-8134e72f-80d2-4446-a618-a442b9ef9879'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:39.832714 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8134e72f-80d2-4446-a618-a442b9ef9879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3f5fa68cae34a3ca767fa20993fb52d', 'request_id': 'req-8134e72f-80d2-4446-a618-a442b9ef9879'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:39.878206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8134e72f-80d2-4446-a618-a442b9ef9879 admin admin] The token cache is being invalidate because project d3f5fa68cae34a3ca767fa20993fb52d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:14:39.878914 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8134e72f-80d2-4446-a618-a442b9ef9879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:39.882253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8134e72f-80d2-4446-a618-a442b9ef9879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:39.901766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1111/2224] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:14:39 2022] DELETE /identity/v3/projects/d3f5fa68cae34a3ca767fa20993fb52d => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:14:41.110183 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324dd1e-826d-46d8-ae66-ff57adcf221b service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:41.113648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-e324dd1e-826d-46d8-ae66-ff57adcf221b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:41.171259 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324dd1e-826d-46d8-ae66-ff57adcf221b service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:41.359866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a44b13-ab56-4b2e-b819-fdfe0cbab434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:41.360943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a44b13-ab56-4b2e-b819-fdfe0cbab434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:41.371014 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a44b13-ab56-4b2e-b819-fdfe0cbab434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:41.371014 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a44b13-ab56-4b2e-b819-fdfe0cbab434 admin admin] PATH_INFO: `/v3/users/b75d3725088d4796826da25b53a85180` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:41.372612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a44b13-ab56-4b2e-b819-fdfe0cbab434 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b75d3725088d4796826da25b53a85180)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:41.377354 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a44b13-ab56-4b2e-b819-fdfe0cbab434 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:41.571568 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-54a44b13-ab56-4b2e-b819-fdfe0cbab434 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b75d3725088d4796826da25b53a85180'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:41.610675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:41.636754 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-54a44b13-ab56-4b2e-b819-fdfe0cbab434 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b75d3725088d4796826da25b53a85180'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:41.657825 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-54a44b13-ab56-4b2e-b819-fdfe0cbab434 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b75d3725088d4796826da25b53a85180'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:41.682014 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-54a44b13-ab56-4b2e-b819-fdfe0cbab434 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b75d3725088d4796826da25b53a85180'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:41.683228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-54a44b13-ab56-4b2e-b819-fdfe0cbab434 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b75d3725088d4796826da25b53a85180'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:41.695186 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1114/2225] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:14:41 2022] DELETE /identity/v3/users/b75d3725088d4796826da25b53a85180 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:14:41.705695 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dcae08-d3af-4bc0-9932-f84d4f199bc5 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:41.705695 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dcae08-d3af-4bc0-9932-f84d4f199bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:41.705695 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dcae08-d3af-4bc0-9932-f84d4f199bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:41.736563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-16dcae08-d3af-4bc0-9932-f84d4f199bc5 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:41.737864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-16dcae08-d3af-4bc0-9932-f84d4f199bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:41.832179 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1112/2226] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:14:41.843871 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a44b13-ab56-4b2e-b819-fdfe0cbab434 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:41.846638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-54a44b13-ab56-4b2e-b819-fdfe0cbab434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:41.969147 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a44b13-ab56-4b2e-b819-fdfe0cbab434 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:41.975327 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3faa17-a598-441b-8793-0d3b38ccb5be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:41.976526 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3faa17-a598-441b-8793-0d3b38ccb5be service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:41.977051 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3faa17-a598-441b-8793-0d3b38ccb5be service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:41.977524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3faa17-a598-441b-8793-0d3b38ccb5be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:41.996787 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3faa17-a598-441b-8793-0d3b38ccb5be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:41.999496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3faa17-a598-441b-8793-0d3b38ccb5be service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:42.027137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1115/2227] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:14:42.361631 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dcae08-d3af-4bc0-9932-f84d4f199bc5 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:42.363596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-16dcae08-d3af-4bc0-9932-f84d4f199bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:42.383290 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dcae08-d3af-4bc0-9932-f84d4f199bc5 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:42.388641 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c5175-e623-44b3-99c8-728ac42e860a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:42.389389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c5175-e623-44b3-99c8-728ac42e860a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:42.389616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c5175-e623-44b3-99c8-728ac42e860a admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:42.390041 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c5175-e623-44b3-99c8-728ac42e860a admin admin] PATH_INFO: `/v3/projects/20371057dd0e41c6988ab054b705b886` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:42.391199 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8c5175-e623-44b3-99c8-728ac42e860a admin admin] RBAC: Authorizing `identity:delete_project(project_id=20371057dd0e41c6988ab054b705b886)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:42.393137 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8c5175-e623-44b3-99c8-728ac42e860a admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:42.463219 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8b8c5175-e623-44b3-99c8-728ac42e860a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20371057dd0e41c6988ab054b705b886', 'request_id': 'req-8b8c5175-e623-44b3-99c8-728ac42e860a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:42.470525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8b8c5175-e623-44b3-99c8-728ac42e860a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20371057dd0e41c6988ab054b705b886', 'request_id': 'req-8b8c5175-e623-44b3-99c8-728ac42e860a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:42.484608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8b8c5175-e623-44b3-99c8-728ac42e860a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20371057dd0e41c6988ab054b705b886', 'request_id': 'req-8b8c5175-e623-44b3-99c8-728ac42e860a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:42.503111 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8b8c5175-e623-44b3-99c8-728ac42e860a admin admin] The token cache is being invalidate because project 20371057dd0e41c6988ab054b705b886 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:14:42.503440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8b8c5175-e623-44b3-99c8-728ac42e860a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:42.504369 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8b8c5175-e623-44b3-99c8-728ac42e860a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:42.506745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1113/2228] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:14:42 2022] DELETE /identity/v3/projects/20371057dd0e41c6988ab054b705b886 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:42.512481 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3faa17-a598-441b-8793-0d3b38ccb5be service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:42.524335 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3faa17-a598-441b-8793-0d3b38ccb5be service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:42.619122 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a0988-e08d-4bf0-aa66-30573266fe28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:42.620114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a0988-e08d-4bf0-aa66-30573266fe28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:42.620381 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a0988-e08d-4bf0-aa66-30573266fe28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:42.620841 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a0988-e08d-4bf0-aa66-30573266fe28 admin admin] PATH_INFO: `/v3/users/7dadc57e7dc1410889a5692b36800895` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:42.622883 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a0988-e08d-4bf0-aa66-30573266fe28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dadc57e7dc1410889a5692b36800895)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:42.625636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a0988-e08d-4bf0-aa66-30573266fe28 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:42.726181 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1f6a0988-e08d-4bf0-aa66-30573266fe28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dadc57e7dc1410889a5692b36800895'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:42.779035 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1f6a0988-e08d-4bf0-aa66-30573266fe28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dadc57e7dc1410889a5692b36800895'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:42.794052 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1f6a0988-e08d-4bf0-aa66-30573266fe28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dadc57e7dc1410889a5692b36800895'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:42.810191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:42.813268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1f6a0988-e08d-4bf0-aa66-30573266fe28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dadc57e7dc1410889a5692b36800895'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:42.816257 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1f6a0988-e08d-4bf0-aa66-30573266fe28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dadc57e7dc1410889a5692b36800895'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:42.836100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1116/2229] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:14:42 2022] DELETE /identity/v3/users/7dadc57e7dc1410889a5692b36800895 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:42.856882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c5175-e623-44b3-99c8-728ac42e860a admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:42.859348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-8b8c5175-e623-44b3-99c8-728ac42e860a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:42.874795 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c5175-e623-44b3-99c8-728ac42e860a admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:42.950414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:42.969352 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a4729-ae1d-49d5-ad80-dd1e4dba0614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:42.970374 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a4729-ae1d-49d5-ad80-dd1e4dba0614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:42.970795 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a4729-ae1d-49d5-ad80-dd1e4dba0614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:42.971276 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a4729-ae1d-49d5-ad80-dd1e4dba0614 admin admin] PATH_INFO: `/v3/users/631741079603419581c44c0bcb560b7f` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:42.973270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a4729-ae1d-49d5-ad80-dd1e4dba0614 admin admin] RBAC: Authorizing `identity:delete_user(user_id=631741079603419581c44c0bcb560b7f)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:42.975535 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a4729-ae1d-49d5-ad80-dd1e4dba0614 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:43.051418 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-cf4a4729-ae1d-49d5-ad80-dd1e4dba0614 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '631741079603419581c44c0bcb560b7f'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:43.066302 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-cf4a4729-ae1d-49d5-ad80-dd1e4dba0614 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '631741079603419581c44c0bcb560b7f'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:43.099076 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-cf4a4729-ae1d-49d5-ad80-dd1e4dba0614 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '631741079603419581c44c0bcb560b7f'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:43.137819 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-cf4a4729-ae1d-49d5-ad80-dd1e4dba0614 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '631741079603419581c44c0bcb560b7f'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:43.137819 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-cf4a4729-ae1d-49d5-ad80-dd1e4dba0614 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '631741079603419581c44c0bcb560b7f'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:43.146250 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1114/2230] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:14:42 2022] DELETE /identity/v3/users/631741079603419581c44c0bcb560b7f => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:14:43.152217 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37442a-27d6-4900-93ed-9b630120d5c5 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:43.152715 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37442a-27d6-4900-93ed-9b630120d5c5 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:43.153235 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37442a-27d6-4900-93ed-9b630120d5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:43.181102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-4c37442a-27d6-4900-93ed-9b630120d5c5 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:43.184083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-4c37442a-27d6-4900-93ed-9b630120d5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:43.324888 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1117/2231] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:43.359493 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a4729-ae1d-49d5-ad80-dd1e4dba0614 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:43.372054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-cf4a4729-ae1d-49d5-ad80-dd1e4dba0614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:43.443299 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37442a-27d6-4900-93ed-9b630120d5c5 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:43.446133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-4c37442a-27d6-4900-93ed-9b630120d5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:43.487848 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a4729-ae1d-49d5-ad80-dd1e4dba0614 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:43.492619 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d8ef3-90e7-4ace-8a40-6bd4303c41f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:43.493266 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d8ef3-90e7-4ace-8a40-6bd4303c41f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:43.493552 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d8ef3-90e7-4ace-8a40-6bd4303c41f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:43.493893 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d8ef3-90e7-4ace-8a40-6bd4303c41f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:43.505378 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d8ef3-90e7-4ace-8a40-6bd4303c41f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:43.506950 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d8ef3-90e7-4ace-8a40-6bd4303c41f4 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:43.512050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37442a-27d6-4900-93ed-9b630120d5c5 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:43.519417 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1115/2232] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:14:43.535309 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a77892-7291-45f4-80ab-565d5bfc1d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:43.536543 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a77892-7291-45f4-80ab-565d5bfc1d5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:43.537131 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a77892-7291-45f4-80ab-565d5bfc1d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:43.537824 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a77892-7291-45f4-80ab-565d5bfc1d5c admin admin] PATH_INFO: `/v3/projects/1749f04b1a4a4b66a3ae50fd5be8f9d3` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:43.539980 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a77892-7291-45f4-80ab-565d5bfc1d5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1749f04b1a4a4b66a3ae50fd5be8f9d3)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:43.545670 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a77892-7291-45f4-80ab-565d5bfc1d5c admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:43.650832 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-55a77892-7291-45f4-80ab-565d5bfc1d5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1749f04b1a4a4b66a3ae50fd5be8f9d3', 'request_id': 'req-55a77892-7291-45f4-80ab-565d5bfc1d5c'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:43.663158 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-55a77892-7291-45f4-80ab-565d5bfc1d5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1749f04b1a4a4b66a3ae50fd5be8f9d3', 'request_id': 'req-55a77892-7291-45f4-80ab-565d5bfc1d5c'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:43.678880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-55a77892-7291-45f4-80ab-565d5bfc1d5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1749f04b1a4a4b66a3ae50fd5be8f9d3', 'request_id': 'req-55a77892-7291-45f4-80ab-565d5bfc1d5c'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:43.709136 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-55a77892-7291-45f4-80ab-565d5bfc1d5c admin admin] The token cache is being invalidate because project 1749f04b1a4a4b66a3ae50fd5be8f9d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:14:43.710046 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-55a77892-7291-45f4-80ab-565d5bfc1d5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:43.713402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-55a77892-7291-45f4-80ab-565d5bfc1d5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:43.721438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1118/2233] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:14:43 2022] DELETE /identity/v3/projects/1749f04b1a4a4b66a3ae50fd5be8f9d3 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:43.723303 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449b202-c573-43f0-87bb-b1d0aba37a5f None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:43.723605 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449b202-c573-43f0-87bb-b1d0aba37a5f None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:43.723974 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449b202-c573-43f0-87bb-b1d0aba37a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:43.771332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-2449b202-c573-43f0-87bb-b1d0aba37a5f None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:43.773913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-2449b202-c573-43f0-87bb-b1d0aba37a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:43.933617 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1116/2234] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:14:43.938234 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96dc76-09f2-4b91-a566-d495721c49af None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:43.938726 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96dc76-09f2-4b91-a566-d495721c49af None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:43.939217 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96dc76-09f2-4b91-a566-d495721c49af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:43.975227 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-3c96dc76-09f2-4b91-a566-d495721c49af None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:43.977321 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-3c96dc76-09f2-4b91-a566-d495721c49af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:43.991926 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1119/2235] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:43.996232 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449b202-c573-43f0-87bb-b1d0aba37a5f None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:44.007675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449b202-c573-43f0-87bb-b1d0aba37a5f None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:44.013276 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea599b9-0530-48e5-9236-e97d70e7dd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:44.014151 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea599b9-0530-48e5-9236-e97d70e7dd74 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:44.014530 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea599b9-0530-48e5-9236-e97d70e7dd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:44.015083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea599b9-0530-48e5-9236-e97d70e7dd74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:44.015839 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea599b9-0530-48e5-9236-e97d70e7dd74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:14:44.016725 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea599b9-0530-48e5-9236-e97d70e7dd74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:44.018545 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea599b9-0530-48e5-9236-e97d70e7dd74 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:44.027123 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:44.051345 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1117/2236] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:14:44.057584 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c96dc76-09f2-4b91-a566-d495721c49af None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:44.074276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c96dc76-09f2-4b91-a566-d495721c49af None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:44.086977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9079e1-fac8-4dce-bc4f-3cc042dffc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:44.088014 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9079e1-fac8-4dce-bc4f-3cc042dffc6a admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:44.089161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9079e1-fac8-4dce-bc4f-3cc042dffc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:44.089650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9079e1-fac8-4dce-bc4f-3cc042dffc6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:44.091058 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9079e1-fac8-4dce-bc4f-3cc042dffc6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:44.095982 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9079e1-fac8-4dce-bc4f-3cc042dffc6a admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:44.163782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1120/2237] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:14:44 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:14:44.168401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea599b9-0530-48e5-9236-e97d70e7dd74 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:44.194466 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea599b9-0530-48e5-9236-e97d70e7dd74 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:44.317657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5723e-3c7e-41f9-982a-07b5512b3195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:44.318344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5723e-3c7e-41f9-982a-07b5512b3195 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:44.318692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5723e-3c7e-41f9-982a-07b5512b3195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:44.319039 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5723e-3c7e-41f9-982a-07b5512b3195 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:44.319789 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5723e-3c7e-41f9-982a-07b5512b3195 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:44.321969 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5723e-3c7e-41f9-982a-07b5512b3195 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:44.360537 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1118/2238] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:14:44 2022] POST /identity/v3/users => generated 291 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:14:44.366259 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9079e1-fac8-4dce-bc4f-3cc042dffc6a admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:44.383106 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9079e1-fac8-4dce-bc4f-3cc042dffc6a admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:44.392529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36869fd8-6fd7-4f5e-a62c-1858dced5cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:44.393395 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36869fd8-6fd7-4f5e-a62c-1858dced5cac admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:44.393670 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36869fd8-6fd7-4f5e-a62c-1858dced5cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:44.393971 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36869fd8-6fd7-4f5e-a62c-1858dced5cac admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:44.394644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36869fd8-6fd7-4f5e-a62c-1858dced5cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:44.396344 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36869fd8-6fd7-4f5e-a62c-1858dced5cac admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:44.411271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1121/2239] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:44.415737 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5723e-3c7e-41f9-982a-07b5512b3195 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:44.430789 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5723e-3c7e-41f9-982a-07b5512b3195 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:44.438896 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e6cb7-943e-4ac6-be61-5529e1d92d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:44.440171 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e6cb7-943e-4ac6-be61-5529e1d92d4b admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:44.440721 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e6cb7-943e-4ac6-be61-5529e1d92d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:44.441154 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e6cb7-943e-4ac6-be61-5529e1d92d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:44.442424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e6cb7-943e-4ac6-be61-5529e1d92d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:44.444426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e6cb7-943e-4ac6-be61-5529e1d92d4b admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:44.461766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1119/2240] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:44.468908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36869fd8-6fd7-4f5e-a62c-1858dced5cac admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:44.485327 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36869fd8-6fd7-4f5e-a62c-1858dced5cac admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:44.491833 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce750c-88f7-4dee-a2fd-5b03291666cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:44.492739 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce750c-88f7-4dee-a2fd-5b03291666cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:44.493193 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce750c-88f7-4dee-a2fd-5b03291666cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:44.493611 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce750c-88f7-4dee-a2fd-5b03291666cf admin admin] PATH_INFO: `/v3/projects/f4a0798f11fc43e4a2fbd7e6d1ba31dc/users/8ec81a8c4b6f4796be3cc606562b10e5/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:44.524259 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce750c-88f7-4dee-a2fd-5b03291666cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4a0798f11fc43e4a2fbd7e6d1ba31dc, user_id=8ec81a8c4b6f4796be3cc606562b10e5, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:44.530366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce750c-88f7-4dee-a2fd-5b03291666cf admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:44.552397 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1122/2241] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:14:44 2022] PUT /identity/v3/projects/f4a0798f11fc43e4a2fbd7e6d1ba31dc/users/8ec81a8c4b6f4796be3cc606562b10e5/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:44.566811 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d78cd5-caf4-49b2-949b-15567ffc372e None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:44.567115 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d78cd5-caf4-49b2-949b-15567ffc372e None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:44.567357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d78cd5-caf4-49b2-949b-15567ffc372e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:44.608946 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-36d78cd5-caf4-49b2-949b-15567ffc372e None None] MFA Rules not processed for user `8ec81a8c4b6f4796be3cc606562b10e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:44.610810 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-36d78cd5-caf4-49b2-949b-15567ffc372e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:44.724800 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1120/2242] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:14:44.729454 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce750c-88f7-4dee-a2fd-5b03291666cf admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:44.766096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:44.849768 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce750c-88f7-4dee-a2fd-5b03291666cf admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:44.857897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d78cd5-caf4-49b2-949b-15567ffc372e None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:44.859855 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14bbf1-d4fe-4e1d-b8a0-7835d074cef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:44.860069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-36d78cd5-caf4-49b2-949b-15567ffc372e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:44.861116 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14bbf1-d4fe-4e1d-b8a0-7835d074cef9 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:44.861676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14bbf1-d4fe-4e1d-b8a0-7835d074cef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:44.862121 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14bbf1-d4fe-4e1d-b8a0-7835d074cef9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:44.863017 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14bbf1-d4fe-4e1d-b8a0-7835d074cef9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:44.866357 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14bbf1-d4fe-4e1d-b8a0-7835d074cef9 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:44.880956 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d78cd5-caf4-49b2-949b-15567ffc372e None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:44.889522 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992608f-dbde-40e7-8150-6cc71f364083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:44.893647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992608f-dbde-40e7-8150-6cc71f364083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:44.895362 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992608f-dbde-40e7-8150-6cc71f364083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:44.896453 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992608f-dbde-40e7-8150-6cc71f364083 admin admin] PATH_INFO: `/v3/projects/89a2efd044d546e9b4648892a85e4b48` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:44.898703 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992608f-dbde-40e7-8150-6cc71f364083 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89a2efd044d546e9b4648892a85e4b48)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:44.902007 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992608f-dbde-40e7-8150-6cc71f364083 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:44.924100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1123/2243] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:14:44 2022] POST /identity/v3/projects => generated 381 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:14:44.926020 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14bbf1-d4fe-4e1d-b8a0-7835d074cef9 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:44.985364 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d992608f-dbde-40e7-8150-6cc71f364083 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89a2efd044d546e9b4648892a85e4b48', 'request_id': 'req-d992608f-dbde-40e7-8150-6cc71f364083'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:44.994614 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d992608f-dbde-40e7-8150-6cc71f364083 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89a2efd044d546e9b4648892a85e4b48', 'request_id': 'req-d992608f-dbde-40e7-8150-6cc71f364083'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:45.000242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:45.007326 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d992608f-dbde-40e7-8150-6cc71f364083 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89a2efd044d546e9b4648892a85e4b48', 'request_id': 'req-d992608f-dbde-40e7-8150-6cc71f364083'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:45.034370 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d992608f-dbde-40e7-8150-6cc71f364083 admin admin] The token cache is being invalidate because project 89a2efd044d546e9b4648892a85e4b48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:14:45.034672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14bbf1-d4fe-4e1d-b8a0-7835d074cef9 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:45.035102 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d992608f-dbde-40e7-8150-6cc71f364083 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:45.035937 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d992608f-dbde-40e7-8150-6cc71f364083 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:45.040309 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1121/2244] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:14:44 2022] DELETE /identity/v3/projects/89a2efd044d546e9b4648892a85e4b48 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:45.043061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992608f-dbde-40e7-8150-6cc71f364083 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:45.045348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d992608f-dbde-40e7-8150-6cc71f364083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:45.197059 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304dc77a-6eaf-4cd8-96c5-813a0d1307a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:45.197587 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304dc77a-6eaf-4cd8-96c5-813a0d1307a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:45.197892 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304dc77a-6eaf-4cd8-96c5-813a0d1307a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:45.198160 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304dc77a-6eaf-4cd8-96c5-813a0d1307a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:45.198807 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304dc77a-6eaf-4cd8-96c5-813a0d1307a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:45.200987 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304dc77a-6eaf-4cd8-96c5-813a0d1307a2 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:45.259607 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992608f-dbde-40e7-8150-6cc71f364083 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:45.259817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1124/2245] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:14:44 2022] POST /identity/v3/users => generated 297 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:14:45.261583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304dc77a-6eaf-4cd8-96c5-813a0d1307a2 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:45.264401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-304dc77a-6eaf-4cd8-96c5-813a0d1307a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:45.270000 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3549c1-115f-445d-8a8d-dbf272c1cdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:45.271207 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3549c1-115f-445d-8a8d-dbf272c1cdcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:45.272071 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3549c1-115f-445d-8a8d-dbf272c1cdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:45.273512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3549c1-115f-445d-8a8d-dbf272c1cdcf admin admin] PATH_INFO: `/v3/users/53b2cdb3b83f4265b650bac712838532` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:45.276723 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3549c1-115f-445d-8a8d-dbf272c1cdcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=53b2cdb3b83f4265b650bac712838532)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:45.282212 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3549c1-115f-445d-8a8d-dbf272c1cdcf admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:45.284618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304dc77a-6eaf-4cd8-96c5-813a0d1307a2 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:45.290641 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a865c94-2b0f-4c86-a353-391b3776a0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:45.291536 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a865c94-2b0f-4c86-a353-391b3776a0a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:45.292110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a865c94-2b0f-4c86-a353-391b3776a0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:45.292464 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a865c94-2b0f-4c86-a353-391b3776a0a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:45.293211 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a865c94-2b0f-4c86-a353-391b3776a0a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:45.295056 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a865c94-2b0f-4c86-a353-391b3776a0a3 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:45.309356 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1125/2246] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:45.316146 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a865c94-2b0f-4c86-a353-391b3776a0a3 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:45.330644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a865c94-2b0f-4c86-a353-391b3776a0a3 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:45.339689 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847c301-1e64-494a-95d9-5e355c44855c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:45.341056 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847c301-1e64-494a-95d9-5e355c44855c admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:45.341668 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847c301-1e64-494a-95d9-5e355c44855c admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:45.342902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847c301-1e64-494a-95d9-5e355c44855c admin admin] PATH_INFO: `/v3/projects/7829f22b9b9c4d50ac2cf9d409a95a32/users/debe02b2c63c41f1a25234603468e77e/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:45.370652 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f847c301-1e64-494a-95d9-5e355c44855c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7829f22b9b9c4d50ac2cf9d409a95a32, user_id=debe02b2c63c41f1a25234603468e77e, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:45.372431 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-3f3549c1-115f-445d-8a8d-dbf272c1cdcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53b2cdb3b83f4265b650bac712838532'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:45.375811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f847c301-1e64-494a-95d9-5e355c44855c admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:45.384848 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-3f3549c1-115f-445d-8a8d-dbf272c1cdcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53b2cdb3b83f4265b650bac712838532'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:45.400523 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1126/2247] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:14:45 2022] PUT /identity/v3/projects/7829f22b9b9c4d50ac2cf9d409a95a32/users/debe02b2c63c41f1a25234603468e77e/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:45.405868 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847c301-1e64-494a-95d9-5e355c44855c admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:45.411387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-3f3549c1-115f-445d-8a8d-dbf272c1cdcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53b2cdb3b83f4265b650bac712838532'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:45.416876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:45.435586 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-3f3549c1-115f-445d-8a8d-dbf272c1cdcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53b2cdb3b83f4265b650bac712838532'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:45.436941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-3f3549c1-115f-445d-8a8d-dbf272c1cdcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53b2cdb3b83f4265b650bac712838532'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:45.453276 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1122/2248] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:14:45 2022] DELETE /identity/v3/users/53b2cdb3b83f4265b650bac712838532 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:14:45.525755 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847c301-1e64-494a-95d9-5e355c44855c admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:45.609644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa7356-a8a2-4ac1-a402-3c95faf50d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:45.610440 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa7356-a8a2-4ac1-a402-3c95faf50d96 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:45.611076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa7356-a8a2-4ac1-a402-3c95faf50d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:45.611076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa7356-a8a2-4ac1-a402-3c95faf50d96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:45.611519 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa7356-a8a2-4ac1-a402-3c95faf50d96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:45.613486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa7356-a8a2-4ac1-a402-3c95faf50d96 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:45.629280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1127/2249] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:45.630445 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3549c1-115f-445d-8a8d-dbf272c1cdcf admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:45.632155 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-3f3549c1-115f-445d-8a8d-dbf272c1cdcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:45.649493 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3549c1-115f-445d-8a8d-dbf272c1cdcf admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:45.657478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98d71d-dc42-45bb-b62c-6d0836dabec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:45.658458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98d71d-dc42-45bb-b62c-6d0836dabec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:45.658894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98d71d-dc42-45bb-b62c-6d0836dabec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:45.659481 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98d71d-dc42-45bb-b62c-6d0836dabec4 admin admin] PATH_INFO: `/v3/projects/7829f22b9b9c4d50ac2cf9d409a95a32/users/debe02b2c63c41f1a25234603468e77e/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:45.663461 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98d71d-dc42-45bb-b62c-6d0836dabec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7829f22b9b9c4d50ac2cf9d409a95a32, user_id=debe02b2c63c41f1a25234603468e77e, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:45.667460 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98d71d-dc42-45bb-b62c-6d0836dabec4 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:45.683098 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1123/2250] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:14:45 2022] PUT /identity/v3/projects/7829f22b9b9c4d50ac2cf9d409a95a32/users/debe02b2c63c41f1a25234603468e77e/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:45.695687 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9d23a-d1c7-4d84-8bd0-199b32ec2a98 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:45.697581 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9d23a-d1c7-4d84-8bd0-199b32ec2a98 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:45.698253 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9d23a-d1c7-4d84-8bd0-199b32ec2a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:45.749085 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-76e9d23a-d1c7-4d84-8bd0-199b32ec2a98 None None] MFA Rules not processed for user `debe02b2c63c41f1a25234603468e77e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:45.750132 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-76e9d23a-d1c7-4d84-8bd0-199b32ec2a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:45.862621 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1128/2251] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:45 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:45.884484 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98d71d-dc42-45bb-b62c-6d0836dabec4 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:45.888620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-ba98d71d-dc42-45bb-b62c-6d0836dabec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:45.991353 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98d71d-dc42-45bb-b62c-6d0836dabec4 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:45.997674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc9448-983e-4b38-85aa-6ccefe517d5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:45.998551 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc9448-983e-4b38-85aa-6ccefe517d5e service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:45.998934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc9448-983e-4b38-85aa-6ccefe517d5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:45.999417 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc9448-983e-4b38-85aa-6ccefe517d5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:46.011867 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:46.017992 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc9448-983e-4b38-85aa-6ccefe517d5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:46.021673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc9448-983e-4b38-85aa-6ccefe517d5e service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:46.046407 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1124/2252] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:45 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:14:46.060167 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9d23a-d1c7-4d84-8bd0-199b32ec2a98 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:46.063257 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-76e9d23a-d1c7-4d84-8bd0-199b32ec2a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:46.186524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:46.292557 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9d23a-d1c7-4d84-8bd0-199b32ec2a98 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:46.322111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd51e7-a593-4ea1-b26a-11c450a0917a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:46.322111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd51e7-a593-4ea1-b26a-11c450a0917a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:46.322111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd51e7-a593-4ea1-b26a-11c450a0917a admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:46.322111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd51e7-a593-4ea1-b26a-11c450a0917a admin admin] PATH_INFO: `/v3/projects/8dfc94da805c47a2887f1aec130e7280` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:46.322727 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd51e7-a593-4ea1-b26a-11c450a0917a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dfc94da805c47a2887f1aec130e7280)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:46.339713 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd51e7-a593-4ea1-b26a-11c450a0917a admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:46.518814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-06dd51e7-a593-4ea1-b26a-11c450a0917a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dfc94da805c47a2887f1aec130e7280', 'request_id': 'req-06dd51e7-a593-4ea1-b26a-11c450a0917a'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:46.540404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-06dd51e7-a593-4ea1-b26a-11c450a0917a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dfc94da805c47a2887f1aec130e7280', 'request_id': 'req-06dd51e7-a593-4ea1-b26a-11c450a0917a'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:46.562053 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-06dd51e7-a593-4ea1-b26a-11c450a0917a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dfc94da805c47a2887f1aec130e7280', 'request_id': 'req-06dd51e7-a593-4ea1-b26a-11c450a0917a'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:46.605726 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-06dd51e7-a593-4ea1-b26a-11c450a0917a admin admin] The token cache is being invalidate because project 8dfc94da805c47a2887f1aec130e7280 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:14:46.606239 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-06dd51e7-a593-4ea1-b26a-11c450a0917a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:46.607436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-06dd51e7-a593-4ea1-b26a-11c450a0917a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:46.612041 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1129/2253] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:14:46 2022] DELETE /identity/v3/projects/8dfc94da805c47a2887f1aec130e7280 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:14:46.618108 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc9448-983e-4b38-85aa-6ccefe517d5e service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:46.641068 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc9448-983e-4b38-85aa-6ccefe517d5e service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:46.806870 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c351b5-e60a-4ade-8d9a-b7d5ddb815ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:46.808222 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c351b5-e60a-4ade-8d9a-b7d5ddb815ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:46.808712 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c351b5-e60a-4ade-8d9a-b7d5ddb815ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:46.809260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c351b5-e60a-4ade-8d9a-b7d5ddb815ea admin admin] PATH_INFO: `/v3/users/6814d8f5f70242a4b797c4a1842471d0` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:46.812193 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c351b5-e60a-4ade-8d9a-b7d5ddb815ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=6814d8f5f70242a4b797c4a1842471d0)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:46.815811 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c351b5-e60a-4ade-8d9a-b7d5ddb815ea admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:46.938792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-16c351b5-e60a-4ade-8d9a-b7d5ddb815ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6814d8f5f70242a4b797c4a1842471d0'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:46.964829 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd51e7-a593-4ea1-b26a-11c450a0917a admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:46.967807 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-06dd51e7-a593-4ea1-b26a-11c450a0917a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:46.968000 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-16c351b5-e60a-4ade-8d9a-b7d5ddb815ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6814d8f5f70242a4b797c4a1842471d0'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:46.999495 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-16c351b5-e60a-4ade-8d9a-b7d5ddb815ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6814d8f5f70242a4b797c4a1842471d0'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:47.018192 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-16c351b5-e60a-4ade-8d9a-b7d5ddb815ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6814d8f5f70242a4b797c4a1842471d0'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:47.019598 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-16c351b5-e60a-4ade-8d9a-b7d5ddb815ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6814d8f5f70242a4b797c4a1842471d0'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:47.036042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1125/2254] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:14:46 2022] DELETE /identity/v3/users/6814d8f5f70242a4b797c4a1842471d0 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:47.101889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd51e7-a593-4ea1-b26a-11c450a0917a admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:47.226314 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f5de8-22ab-429e-9ce1-0f01e453f7a4 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'debe02b2c63c41f1a25234603468e77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7829f22b9b9c4d50ac2cf9d409a95a32', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:47.227823 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f5de8-22ab-429e-9ce1-0f01e453f7a4 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:47.229759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f5de8-22ab-429e-9ce1-0f01e453f7a4 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:47.230457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f5de8-22ab-429e-9ce1-0f01e453f7a4 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:47.232093 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f5de8-22ab-429e-9ce1-0f01e453f7a4 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:14:47.233513 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f5de8-22ab-429e-9ce1-0f01e453f7a4 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:47.237971 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f5de8-22ab-429e-9ce1-0f01e453f7a4 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:47.263436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1130/2255] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:14:47.269828 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c351b5-e60a-4ade-8d9a-b7d5ddb815ea admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:47.273183 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-16c351b5-e60a-4ade-8d9a-b7d5ddb815ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:47.297332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c351b5-e60a-4ade-8d9a-b7d5ddb815ea admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:47.303872 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1f545-60aa-483a-8255-b000780c9a81 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'debe02b2c63c41f1a25234603468e77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7829f22b9b9c4d50ac2cf9d409a95a32', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:47.304673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1f545-60aa-483a-8255-b000780c9a81 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:47.304954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1f545-60aa-483a-8255-b000780c9a81 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:47.306770 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1f545-60aa-483a-8255-b000780c9a81 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:47.306770 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1f545-60aa-483a-8255-b000780c9a81 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:47.311543 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1f545-60aa-483a-8255-b000780c9a81 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:47.361391 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1126/2256] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:14:47 2022] POST /identity/v3/projects => generated 355 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:14:47.365746 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f5de8-22ab-429e-9ce1-0f01e453f7a4 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:47.376967 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:47.385479 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f5de8-22ab-429e-9ce1-0f01e453f7a4 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:47.499957 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1288997-9a4f-40f6-840c-29e047236d96 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'debe02b2c63c41f1a25234603468e77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7829f22b9b9c4d50ac2cf9d409a95a32', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:47.501145 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1288997-9a4f-40f6-840c-29e047236d96 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:47.501776 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1288997-9a4f-40f6-840c-29e047236d96 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:47.502357 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1288997-9a4f-40f6-840c-29e047236d96 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:47.503631 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1288997-9a4f-40f6-840c-29e047236d96 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:14:47.504286 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1288997-9a4f-40f6-840c-29e047236d96 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:47.506969 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1288997-9a4f-40f6-840c-29e047236d96 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:47.539817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1131/2257] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:14:47.573029 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1f545-60aa-483a-8255-b000780c9a81 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:47.588073 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1f545-60aa-483a-8255-b000780c9a81 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:47.594297 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e57b78-e06e-4e4f-83c7-ec324e09d8bf tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'debe02b2c63c41f1a25234603468e77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7829f22b9b9c4d50ac2cf9d409a95a32', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:47.597387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e57b78-e06e-4e4f-83c7-ec324e09d8bf tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:47.597796 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e57b78-e06e-4e4f-83c7-ec324e09d8bf tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:47.598304 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e57b78-e06e-4e4f-83c7-ec324e09d8bf tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] PATH_INFO: `/v3/projects/15b62b51cbbf4ee3b19d0cf5248d5a96` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:47.599840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e57b78-e06e-4e4f-83c7-ec324e09d8bf tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] RBAC: Authorizing `identity:delete_project(project_id=15b62b51cbbf4ee3b19d0cf5248d5a96)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:47.602027 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e57b78-e06e-4e4f-83c7-ec324e09d8bf tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:47.688869 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:47.692817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f9e57b78-e06e-4e4f-83c7-ec324e09d8bf tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15b62b51cbbf4ee3b19d0cf5248d5a96', 'request_id': 'req-f9e57b78-e06e-4e4f-83c7-ec324e09d8bf'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:47.708108 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f9e57b78-e06e-4e4f-83c7-ec324e09d8bf tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15b62b51cbbf4ee3b19d0cf5248d5a96', 'request_id': 'req-f9e57b78-e06e-4e4f-83c7-ec324e09d8bf'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:47.734393 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f9e57b78-e06e-4e4f-83c7-ec324e09d8bf tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15b62b51cbbf4ee3b19d0cf5248d5a96', 'request_id': 'req-f9e57b78-e06e-4e4f-83c7-ec324e09d8bf'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:47.755853 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f9e57b78-e06e-4e4f-83c7-ec324e09d8bf tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] The token cache is being invalidate because project 15b62b51cbbf4ee3b19d0cf5248d5a96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:14:47.756134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f9e57b78-e06e-4e4f-83c7-ec324e09d8bf tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:47.756955 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f9e57b78-e06e-4e4f-83c7-ec324e09d8bf tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:47.758837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1127/2258] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:14:47 2022] DELETE /identity/v3/projects/15b62b51cbbf4ee3b19d0cf5248d5a96 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:48.112510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1288997-9a4f-40f6-840c-29e047236d96 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:48.114689 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a1288997-9a4f-40f6-840c-29e047236d96 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:48.138817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e57b78-e06e-4e4f-83c7-ec324e09d8bf tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:48.141148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-f9e57b78-e06e-4e4f-83c7-ec324e09d8bf tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:48.301367 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1288997-9a4f-40f6-840c-29e047236d96 tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:48.309848 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0b546-daae-47d9-9935-473ad37fc014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:48.312493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0b546-daae-47d9-9935-473ad37fc014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:48.313300 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0b546-daae-47d9-9935-473ad37fc014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:48.315546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0b546-daae-47d9-9935-473ad37fc014 admin admin] PATH_INFO: `/v3/projects/995f4ac780b047948405bc727ecbc240` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:48.317517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0b546-daae-47d9-9935-473ad37fc014 admin admin] RBAC: Authorizing `identity:delete_project(project_id=995f4ac780b047948405bc727ecbc240)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:48.321890 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0b546-daae-47d9-9935-473ad37fc014 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:48.324844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e57b78-e06e-4e4f-83c7-ec324e09d8bf tempest-VolumeQuotaClassesTest-1000232630 tempest-VolumeQuotaClassesTest-1000232630-project-admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:48.336530 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3457b3-c245-43d6-b9c8-4fa4e118f573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:48.338077 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3457b3-c245-43d6-b9c8-4fa4e118f573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:48.338706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3457b3-c245-43d6-b9c8-4fa4e118f573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:48.339461 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3457b3-c245-43d6-b9c8-4fa4e118f573 admin admin] PATH_INFO: `/v3/users/8ec81a8c4b6f4796be3cc606562b10e5` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:48.341230 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3457b3-c245-43d6-b9c8-4fa4e118f573 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ec81a8c4b6f4796be3cc606562b10e5)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:48.344925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3457b3-c245-43d6-b9c8-4fa4e118f573 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:48.424545 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-5ce0b546-daae-47d9-9935-473ad37fc014 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '995f4ac780b047948405bc727ecbc240', 'request_id': 'req-5ce0b546-daae-47d9-9935-473ad37fc014'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:48.432544 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-6b3457b3-c245-43d6-b9c8-4fa4e118f573 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ec81a8c4b6f4796be3cc606562b10e5'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:48.434002 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-5ce0b546-daae-47d9-9935-473ad37fc014 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '995f4ac780b047948405bc727ecbc240', 'request_id': 'req-5ce0b546-daae-47d9-9935-473ad37fc014'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:48.444507 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-5ce0b546-daae-47d9-9935-473ad37fc014 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '995f4ac780b047948405bc727ecbc240', 'request_id': 'req-5ce0b546-daae-47d9-9935-473ad37fc014'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:48.455858 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-6b3457b3-c245-43d6-b9c8-4fa4e118f573 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ec81a8c4b6f4796be3cc606562b10e5'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:48.464171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-5ce0b546-daae-47d9-9935-473ad37fc014 admin admin] The token cache is being invalidate because project 995f4ac780b047948405bc727ecbc240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:14:48.465140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-5ce0b546-daae-47d9-9935-473ad37fc014 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:48.465412 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-5ce0b546-daae-47d9-9935-473ad37fc014 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:48.468368 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1132/2259] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:14:48 2022] DELETE /identity/v3/projects/995f4ac780b047948405bc727ecbc240 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:48.475993 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82de9a7-4590-49cf-9753-ceabde709985 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:48.476646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82de9a7-4590-49cf-9753-ceabde709985 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:48.477395 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82de9a7-4590-49cf-9753-ceabde709985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:48.483331 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-6b3457b3-c245-43d6-b9c8-4fa4e118f573 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ec81a8c4b6f4796be3cc606562b10e5'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:48.519655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-6b3457b3-c245-43d6-b9c8-4fa4e118f573 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ec81a8c4b6f4796be3cc606562b10e5'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:48.519950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-f82de9a7-4590-49cf-9753-ceabde709985 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:48.521482 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-6b3457b3-c245-43d6-b9c8-4fa4e118f573 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ec81a8c4b6f4796be3cc606562b10e5'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:48.521809 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f82de9a7-4590-49cf-9753-ceabde709985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:48.537689 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1128/2260] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:14:48 2022] DELETE /identity/v3/users/8ec81a8c4b6f4796be3cc606562b10e5 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:48.542662 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a20094-9cd9-488c-bfdc-f9e58aea805e None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:48.543141 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a20094-9cd9-488c-bfdc-f9e58aea805e None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:48.543590 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a20094-9cd9-488c-bfdc-f9e58aea805e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:48.577742 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:48.583004 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-d2a20094-9cd9-488c-bfdc-f9e58aea805e None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:48.585043 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d2a20094-9cd9-488c-bfdc-f9e58aea805e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:48.659700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1133/2261] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:48.671129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed1f61-6365-4bbf-8003-f14948e29616 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:48.671129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed1f61-6365-4bbf-8003-f14948e29616 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:48.671129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed1f61-6365-4bbf-8003-f14948e29616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:48.679589 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:48.713782 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1129/2262] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:48.725541 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-52ed1f61-6365-4bbf-8003-f14948e29616 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:48.728680 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-52ed1f61-6365-4bbf-8003-f14948e29616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:48.735391 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a20094-9cd9-488c-bfdc-f9e58aea805e None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:48.737623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d2a20094-9cd9-488c-bfdc-f9e58aea805e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:48.771742 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1134/2263] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:14:48.773507 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed1f61-6365-4bbf-8003-f14948e29616 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:48.805633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed1f61-6365-4bbf-8003-f14948e29616 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:48.814540 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d35cd98-4337-4e1f-b869-87f92b42dc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:48.815628 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35cd98-4337-4e1f-b869-87f92b42dc39 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:48.815927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35cd98-4337-4e1f-b869-87f92b42dc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:48.816215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35cd98-4337-4e1f-b869-87f92b42dc39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:48.816998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d35cd98-4337-4e1f-b869-87f92b42dc39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:14:48.817353 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d35cd98-4337-4e1f-b869-87f92b42dc39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:48.819217 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d35cd98-4337-4e1f-b869-87f92b42dc39 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:48.856639 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1135/2264] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:14:48.859646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d35cd98-4337-4e1f-b869-87f92b42dc39 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:48.863218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a20094-9cd9-488c-bfdc-f9e58aea805e None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:48.868556 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2cf51-f345-43e2-a022-7d361e34cccc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:48.869750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2cf51-f345-43e2-a022-7d361e34cccc service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:48.869750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2cf51-f345-43e2-a022-7d361e34cccc service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:48.870063 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2cf51-f345-43e2-a022-7d361e34cccc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:48.896815 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2cf51-f345-43e2-a022-7d361e34cccc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:48.901115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d35cd98-4337-4e1f-b869-87f92b42dc39 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:48.902261 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2cf51-f345-43e2-a022-7d361e34cccc service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:48.910836 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200690d-5274-4d8e-9cab-44e9047abf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:48.912257 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200690d-5274-4d8e-9cab-44e9047abf55 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:48.912926 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200690d-5274-4d8e-9cab-44e9047abf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:48.913390 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200690d-5274-4d8e-9cab-44e9047abf55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:48.916067 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200690d-5274-4d8e-9cab-44e9047abf55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:48.920711 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200690d-5274-4d8e-9cab-44e9047abf55 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:48.928673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1130/2265] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:14:48.980097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1136/2266] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:14:48 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:14:48.984156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2cf51-f345-43e2-a022-7d361e34cccc service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:49.004583 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2cf51-f345-43e2-a022-7d361e34cccc service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:49.113036 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945a34e-5f54-4022-9cd8-7a49b6fbac26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:49.113036 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945a34e-5f54-4022-9cd8-7a49b6fbac26 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:49.113036 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945a34e-5f54-4022-9cd8-7a49b6fbac26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:49.113036 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945a34e-5f54-4022-9cd8-7a49b6fbac26 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:49.114104 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945a34e-5f54-4022-9cd8-7a49b6fbac26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:49.117290 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945a34e-5f54-4022-9cd8-7a49b6fbac26 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:49.154481 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1131/2267] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:14:48 2022] POST /identity/v3/users => generated 292 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:14:49.155659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200690d-5274-4d8e-9cab-44e9047abf55 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:49.168859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200690d-5274-4d8e-9cab-44e9047abf55 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:49.175910 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa3563-aac1-4db8-80eb-6e9ce0f9bbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:49.176540 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa3563-aac1-4db8-80eb-6e9ce0f9bbc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:49.176896 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa3563-aac1-4db8-80eb-6e9ce0f9bbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:49.177212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa3563-aac1-4db8-80eb-6e9ce0f9bbc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:49.180389 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa3563-aac1-4db8-80eb-6e9ce0f9bbc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:49.180389 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa3563-aac1-4db8-80eb-6e9ce0f9bbc3 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:49.201072 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1137/2268] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:14:49.204037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945a34e-5f54-4022-9cd8-7a49b6fbac26 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:49.225558 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945a34e-5f54-4022-9cd8-7a49b6fbac26 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:49.234323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3353a-f1d7-4cf4-bb01-3f3297d7654c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:49.235587 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3353a-f1d7-4cf4-bb01-3f3297d7654c admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:49.236057 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3353a-f1d7-4cf4-bb01-3f3297d7654c admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:49.236506 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3353a-f1d7-4cf4-bb01-3f3297d7654c admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:49.237633 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3353a-f1d7-4cf4-bb01-3f3297d7654c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:49.240616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3353a-f1d7-4cf4-bb01-3f3297d7654c admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:49.257204 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1132/2269] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:49.261668 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa3563-aac1-4db8-80eb-6e9ce0f9bbc3 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:49.274435 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa3563-aac1-4db8-80eb-6e9ce0f9bbc3 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:49.281833 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381cad05-dba9-4f33-a8c3-3bf2ddd6e4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:49.282605 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cad05-dba9-4f33-a8c3-3bf2ddd6e4f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:49.282605 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cad05-dba9-4f33-a8c3-3bf2ddd6e4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:49.282875 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cad05-dba9-4f33-a8c3-3bf2ddd6e4f4 admin admin] PATH_INFO: `/v3/projects/437cf4024a9640bdbe14e3788aaa3ce7/users/ba3bbb4262a94b818bf038b987329b9d/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:49.302938 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381cad05-dba9-4f33-a8c3-3bf2ddd6e4f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=437cf4024a9640bdbe14e3788aaa3ce7, user_id=ba3bbb4262a94b818bf038b987329b9d, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:49.307544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381cad05-dba9-4f33-a8c3-3bf2ddd6e4f4 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:49.329035 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1138/2270] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:14:49 2022] PUT /identity/v3/projects/437cf4024a9640bdbe14e3788aaa3ce7/users/ba3bbb4262a94b818bf038b987329b9d/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:49.346738 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82866750-c809-42ec-9178-ff4ecd1a4c62 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:49.347233 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82866750-c809-42ec-9178-ff4ecd1a4c62 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:49.347803 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82866750-c809-42ec-9178-ff4ecd1a4c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:49.391494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-82866750-c809-42ec-9178-ff4ecd1a4c62 None None] MFA Rules not processed for user `ba3bbb4262a94b818bf038b987329b9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:49.393385 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-82866750-c809-42ec-9178-ff4ecd1a4c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:49.519678 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381cad05-dba9-4f33-a8c3-3bf2ddd6e4f4 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:49.522562 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-381cad05-dba9-4f33-a8c3-3bf2ddd6e4f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:49.545934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1133/2271] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:49.569397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82866750-c809-42ec-9178-ff4ecd1a4c62 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:49.573487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-82866750-c809-42ec-9178-ff4ecd1a4c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:49.594613 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:49.666298 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381cad05-dba9-4f33-a8c3-3bf2ddd6e4f4 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:49.676912 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82866750-c809-42ec-9178-ff4ecd1a4c62 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:49.677130 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55e093-0e9f-4187-a9ca-d9575152b4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:49.678286 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55e093-0e9f-4187-a9ca-d9575152b4a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:49.678788 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55e093-0e9f-4187-a9ca-d9575152b4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:49.679246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55e093-0e9f-4187-a9ca-d9575152b4a2 admin admin] PATH_INFO: `/v3/projects/f4a0798f11fc43e4a2fbd7e6d1ba31dc` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:49.681259 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed55e093-0e9f-4187-a9ca-d9575152b4a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4a0798f11fc43e4a2fbd7e6d1ba31dc)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:49.681897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740a6a7-ad64-487c-96ed-5cdfee624376 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:49.682526 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740a6a7-ad64-487c-96ed-5cdfee624376 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:49.682795 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740a6a7-ad64-487c-96ed-5cdfee624376 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:49.683075 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740a6a7-ad64-487c-96ed-5cdfee624376 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:49.684950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed55e093-0e9f-4187-a9ca-d9575152b4a2 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:49.694812 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f740a6a7-ad64-487c-96ed-5cdfee624376 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:49.698382 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f740a6a7-ad64-487c-96ed-5cdfee624376 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:49.716044 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1134/2272] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:14:49.782429 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-ed55e093-0e9f-4187-a9ca-d9575152b4a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4a0798f11fc43e4a2fbd7e6d1ba31dc', 'request_id': 'req-ed55e093-0e9f-4187-a9ca-d9575152b4a2'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:49.784663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:49.793201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-ed55e093-0e9f-4187-a9ca-d9575152b4a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4a0798f11fc43e4a2fbd7e6d1ba31dc', 'request_id': 'req-ed55e093-0e9f-4187-a9ca-d9575152b4a2'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:49.804808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-ed55e093-0e9f-4187-a9ca-d9575152b4a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4a0798f11fc43e4a2fbd7e6d1ba31dc', 'request_id': 'req-ed55e093-0e9f-4187-a9ca-d9575152b4a2'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:49.824125 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-ed55e093-0e9f-4187-a9ca-d9575152b4a2 admin admin] The token cache is being invalidate because project f4a0798f11fc43e4a2fbd7e6d1ba31dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:14:49.824557 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-ed55e093-0e9f-4187-a9ca-d9575152b4a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:49.825372 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-ed55e093-0e9f-4187-a9ca-d9575152b4a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:49.828089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1139/2273] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:14:49 2022] DELETE /identity/v3/projects/f4a0798f11fc43e4a2fbd7e6d1ba31dc => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:49.835165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740a6a7-ad64-487c-96ed-5cdfee624376 service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:49.854729 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740a6a7-ad64-487c-96ed-5cdfee624376 service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:49.952543 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdddb68-d4ba-409c-bec8-ef6d6cb9b1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:49.953234 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdddb68-d4ba-409c-bec8-ef6d6cb9b1a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:49.953519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdddb68-d4ba-409c-bec8-ef6d6cb9b1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:49.953839 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdddb68-d4ba-409c-bec8-ef6d6cb9b1a1 admin admin] PATH_INFO: `/v3/users/debe02b2c63c41f1a25234603468e77e` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:49.955092 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdddb68-d4ba-409c-bec8-ef6d6cb9b1a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=debe02b2c63c41f1a25234603468e77e)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:49.957257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdddb68-d4ba-409c-bec8-ef6d6cb9b1a1 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:50.052362 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-7fdddb68-d4ba-409c-bec8-ef6d6cb9b1a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'debe02b2c63c41f1a25234603468e77e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:50.067709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-7fdddb68-d4ba-409c-bec8-ef6d6cb9b1a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'debe02b2c63c41f1a25234603468e77e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:50.100024 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-7fdddb68-d4ba-409c-bec8-ef6d6cb9b1a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'debe02b2c63c41f1a25234603468e77e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:50.119440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-7fdddb68-d4ba-409c-bec8-ef6d6cb9b1a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'debe02b2c63c41f1a25234603468e77e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:50.120645 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-7fdddb68-d4ba-409c-bec8-ef6d6cb9b1a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'debe02b2c63c41f1a25234603468e77e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:50.145065 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1135/2274] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:14:49 2022] DELETE /identity/v3/users/debe02b2c63c41f1a25234603468e77e => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:50.747683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55e093-0e9f-4187-a9ca-d9575152b4a2 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:50.750718 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ed55e093-0e9f-4187-a9ca-d9575152b4a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:50.783735 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55e093-0e9f-4187-a9ca-d9575152b4a2 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:50.879723 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34583dcc-09b4-49fd-91cf-7d9deae6a76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:50.880475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34583dcc-09b4-49fd-91cf-7d9deae6a76d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:50.880942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34583dcc-09b4-49fd-91cf-7d9deae6a76d admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:50.882213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34583dcc-09b4-49fd-91cf-7d9deae6a76d admin admin] PATH_INFO: `/v3/projects/7829f22b9b9c4d50ac2cf9d409a95a32` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:50.883912 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34583dcc-09b4-49fd-91cf-7d9deae6a76d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7829f22b9b9c4d50ac2cf9d409a95a32)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:50.887124 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34583dcc-09b4-49fd-91cf-7d9deae6a76d admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:50.970977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-34583dcc-09b4-49fd-91cf-7d9deae6a76d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7829f22b9b9c4d50ac2cf9d409a95a32', 'request_id': 'req-34583dcc-09b4-49fd-91cf-7d9deae6a76d'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:50.981912 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-34583dcc-09b4-49fd-91cf-7d9deae6a76d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7829f22b9b9c4d50ac2cf9d409a95a32', 'request_id': 'req-34583dcc-09b4-49fd-91cf-7d9deae6a76d'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:50.991729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-34583dcc-09b4-49fd-91cf-7d9deae6a76d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7829f22b9b9c4d50ac2cf9d409a95a32', 'request_id': 'req-34583dcc-09b4-49fd-91cf-7d9deae6a76d'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:51.005587 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-34583dcc-09b4-49fd-91cf-7d9deae6a76d admin admin] The token cache is being invalidate because project 7829f22b9b9c4d50ac2cf9d409a95a32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:14:51.005587 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-34583dcc-09b4-49fd-91cf-7d9deae6a76d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:51.006575 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-34583dcc-09b4-49fd-91cf-7d9deae6a76d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:51.007946 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1140/2275] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:14:50 2022] DELETE /identity/v3/projects/7829f22b9b9c4d50ac2cf9d409a95a32 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:51.015772 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b4e2d-6473-487d-90fc-db7cb2b9ddb9 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:51.016330 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b4e2d-6473-487d-90fc-db7cb2b9ddb9 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:51.016449 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b4e2d-6473-487d-90fc-db7cb2b9ddb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:51.041152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-dd1b4e2d-6473-487d-90fc-db7cb2b9ddb9 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:51.042635 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-dd1b4e2d-6473-487d-90fc-db7cb2b9ddb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:51.069455 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:51.130399 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1136/2276] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:51.139536 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ad980-8b7d-4b1b-97fd-f4f13292c834 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:51.140144 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ad980-8b7d-4b1b-97fd-f4f13292c834 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:51.140709 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ad980-8b7d-4b1b-97fd-f4f13292c834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:51.171616 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-334ad980-8b7d-4b1b-97fd-f4f13292c834 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:51.172856 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-334ad980-8b7d-4b1b-97fd-f4f13292c834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:51.187789 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1141/2277] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:51.192989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b4e2d-6473-487d-90fc-db7cb2b9ddb9 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:51.204947 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b4e2d-6473-487d-90fc-db7cb2b9ddb9 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:51.212227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b6015-8001-4e9d-afe3-44501ffc379d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:51.212992 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b6015-8001-4e9d-afe3-44501ffc379d admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:51.213453 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b6015-8001-4e9d-afe3-44501ffc379d admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:51.213956 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b6015-8001-4e9d-afe3-44501ffc379d admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:51.214959 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b6015-8001-4e9d-afe3-44501ffc379d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:14:51.215342 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b6015-8001-4e9d-afe3-44501ffc379d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:51.218719 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b6015-8001-4e9d-afe3-44501ffc379d admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:51.232128 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1137/2278] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:14:51.237979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ad980-8b7d-4b1b-97fd-f4f13292c834 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:51.252755 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ad980-8b7d-4b1b-97fd-f4f13292c834 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:51.260906 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b81911-a7f9-49f0-8ab6-7a87a1fee433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:51.262133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b81911-a7f9-49f0-8ab6-7a87a1fee433 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:51.262739 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b81911-a7f9-49f0-8ab6-7a87a1fee433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:51.263313 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b81911-a7f9-49f0-8ab6-7a87a1fee433 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:51.265235 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b81911-a7f9-49f0-8ab6-7a87a1fee433 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:51.268512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b81911-a7f9-49f0-8ab6-7a87a1fee433 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:51.310847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1142/2279] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:14:51 2022] POST /identity/v3/projects => generated 381 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:14:51.316700 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b6015-8001-4e9d-afe3-44501ffc379d admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:51.343070 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b6015-8001-4e9d-afe3-44501ffc379d admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:51.442908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b4c03-4abd-482a-b177-beca00d91a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:51.443763 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b4c03-4abd-482a-b177-beca00d91a1f admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:51.443943 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b4c03-4abd-482a-b177-beca00d91a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:51.444228 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b4c03-4abd-482a-b177-beca00d91a1f admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:51.444932 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b4c03-4abd-482a-b177-beca00d91a1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:51.446902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b4c03-4abd-482a-b177-beca00d91a1f admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:51.487845 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1138/2280] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:14:51 2022] POST /identity/v3/users => generated 291 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:14:51.491148 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b81911-a7f9-49f0-8ab6-7a87a1fee433 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:51.511942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b81911-a7f9-49f0-8ab6-7a87a1fee433 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:51.522425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0694fb-2fe8-466c-87cf-b3415fb66b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:51.523614 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0694fb-2fe8-466c-87cf-b3415fb66b6b admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:51.524109 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0694fb-2fe8-466c-87cf-b3415fb66b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:51.528191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0694fb-2fe8-466c-87cf-b3415fb66b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:51.528191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0694fb-2fe8-466c-87cf-b3415fb66b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:51.529482 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0694fb-2fe8-466c-87cf-b3415fb66b6b admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:51.550664 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1143/2281] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:51.556981 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b4c03-4abd-482a-b177-beca00d91a1f admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:51.571595 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b4c03-4abd-482a-b177-beca00d91a1f admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:51.578013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8368ab-383c-4299-8793-19e0668e6e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:51.579272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8368ab-383c-4299-8793-19e0668e6e85 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:51.580243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8368ab-383c-4299-8793-19e0668e6e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:51.580614 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8368ab-383c-4299-8793-19e0668e6e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:51.581456 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8368ab-383c-4299-8793-19e0668e6e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:51.583735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8368ab-383c-4299-8793-19e0668e6e85 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:51.611892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1139/2282] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:51.616980 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0694fb-2fe8-466c-87cf-b3415fb66b6b admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:51.634507 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0694fb-2fe8-466c-87cf-b3415fb66b6b admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:51.641168 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064c4a8-4863-4e2a-9abb-5e55c4519fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:51.641914 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064c4a8-4863-4e2a-9abb-5e55c4519fbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:51.642188 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064c4a8-4863-4e2a-9abb-5e55c4519fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:51.642520 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064c4a8-4863-4e2a-9abb-5e55c4519fbc admin admin] PATH_INFO: `/v3/projects/5359d4a4cbfa49b4b2e58d373de21ab8/users/06fbbccaf1e84303abc8bd04cba805bc/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:51.672635 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9064c4a8-4863-4e2a-9abb-5e55c4519fbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5359d4a4cbfa49b4b2e58d373de21ab8, user_id=06fbbccaf1e84303abc8bd04cba805bc, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:51.678226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9064c4a8-4863-4e2a-9abb-5e55c4519fbc admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:51.704441 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1144/2283] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:14:51 2022] PUT /identity/v3/projects/5359d4a4cbfa49b4b2e58d373de21ab8/users/06fbbccaf1e84303abc8bd04cba805bc/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:14:51.718932 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f180a6-9c62-4e99-8d44-d16101a16412 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:51.719435 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f180a6-9c62-4e99-8d44-d16101a16412 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:51.719933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f180a6-9c62-4e99-8d44-d16101a16412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:51.764670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-36f180a6-9c62-4e99-8d44-d16101a16412 None None] MFA Rules not processed for user `06fbbccaf1e84303abc8bd04cba805bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:51.766750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-36f180a6-9c62-4e99-8d44-d16101a16412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:51.919235 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1140/2284] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:51.924764 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064c4a8-4863-4e2a-9abb-5e55c4519fbc admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:51.941985 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:52.042467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064c4a8-4863-4e2a-9abb-5e55c4519fbc admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:52.048096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d996c7-3c25-4959-af4e-26ff859c0697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:52.049498 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d996c7-3c25-4959-af4e-26ff859c0697 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:52.049498 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d996c7-3c25-4959-af4e-26ff859c0697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:52.049850 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d996c7-3c25-4959-af4e-26ff859c0697 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:52.050682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d996c7-3c25-4959-af4e-26ff859c0697 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:52.053178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d996c7-3c25-4959-af4e-26ff859c0697 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:52.112134 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1145/2285] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:14:51 2022] POST /identity/v3/projects => generated 383 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:14:52.114261 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f180a6-9c62-4e99-8d44-d16101a16412 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:52.130721 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f180a6-9c62-4e99-8d44-d16101a16412 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:52.159617 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:52.224190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ad7e5-af95-48b6-825e-10cd50078e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:52.225327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ad7e5-af95-48b6-825e-10cd50078e87 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:52.225689 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ad7e5-af95-48b6-825e-10cd50078e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:52.226148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ad7e5-af95-48b6-825e-10cd50078e87 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:52.227874 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3ad7e5-af95-48b6-825e-10cd50078e87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:52.230275 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3ad7e5-af95-48b6-825e-10cd50078e87 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:52.262987 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1141/2286] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:14:52 2022] POST /identity/v3/users => generated 298 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:14:52.271221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d996c7-3c25-4959-af4e-26ff859c0697 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:52.285473 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d996c7-3c25-4959-af4e-26ff859c0697 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:52.290756 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b0fa2-62c0-4b6d-b958-ae1e15371fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:52.291377 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b0fa2-62c0-4b6d-b958-ae1e15371fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:52.291657 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b0fa2-62c0-4b6d-b958-ae1e15371fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:52.291933 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b0fa2-62c0-4b6d-b958-ae1e15371fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:52.292589 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b0fa2-62c0-4b6d-b958-ae1e15371fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:52.294250 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b0fa2-62c0-4b6d-b958-ae1e15371fa0 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:52.306631 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1146/2287] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:52.313688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ad7e5-af95-48b6-825e-10cd50078e87 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:52.326499 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ad7e5-af95-48b6-825e-10cd50078e87 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:52.333219 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdf15a-75d8-48e2-ae46-b01d4cda5d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:52.334367 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdf15a-75d8-48e2-ae46-b01d4cda5d35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:52.334848 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdf15a-75d8-48e2-ae46-b01d4cda5d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:52.335330 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdf15a-75d8-48e2-ae46-b01d4cda5d35 admin admin] PATH_INFO: `/v3/projects/00777eb7189e48368b572b088cdd77b7/users/314d887dfdc54571894cae67135d021a/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:52.353275 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfdf15a-75d8-48e2-ae46-b01d4cda5d35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00777eb7189e48368b572b088cdd77b7, user_id=314d887dfdc54571894cae67135d021a, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:52.356013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfdf15a-75d8-48e2-ae46-b01d4cda5d35 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:52.378181 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1142/2288] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:14:52 2022] PUT /identity/v3/projects/00777eb7189e48368b572b088cdd77b7/users/314d887dfdc54571894cae67135d021a/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:52.382540 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b0fa2-62c0-4b6d-b958-ae1e15371fa0 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:52.394459 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b0fa2-62c0-4b6d-b958-ae1e15371fa0 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:52.458676 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d4987-bbd3-495a-bfb6-5daf85204697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:52.459451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d4987-bbd3-495a-bfb6-5daf85204697 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:52.459735 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d4987-bbd3-495a-bfb6-5daf85204697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:52.460033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d4987-bbd3-495a-bfb6-5daf85204697 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:52.460655 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d4987-bbd3-495a-bfb6-5daf85204697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:52.462302 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d4987-bbd3-495a-bfb6-5daf85204697 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:52.476396 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1147/2289] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:52.481656 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdf15a-75d8-48e2-ae46-b01d4cda5d35 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:52.495523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdf15a-75d8-48e2-ae46-b01d4cda5d35 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:52.502108 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7dbe26-ec9a-4352-9e7e-98d424cddd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:52.502864 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dbe26-ec9a-4352-9e7e-98d424cddd62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:52.503216 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dbe26-ec9a-4352-9e7e-98d424cddd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:52.503797 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dbe26-ec9a-4352-9e7e-98d424cddd62 admin admin] PATH_INFO: `/v3/projects/00777eb7189e48368b572b088cdd77b7/users/314d887dfdc54571894cae67135d021a/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:52.506003 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7dbe26-ec9a-4352-9e7e-98d424cddd62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00777eb7189e48368b572b088cdd77b7, user_id=314d887dfdc54571894cae67135d021a, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:52.509223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7dbe26-ec9a-4352-9e7e-98d424cddd62 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:52.521582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1143/2290] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:14:52 2022] PUT /identity/v3/projects/00777eb7189e48368b572b088cdd77b7/users/314d887dfdc54571894cae67135d021a/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:14:52.532961 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad458b7-53ff-495e-b28d-ee21ad04dfaa None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:52.533246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad458b7-53ff-495e-b28d-ee21ad04dfaa None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:52.533525 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad458b7-53ff-495e-b28d-ee21ad04dfaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:52.569556 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-1ad458b7-53ff-495e-b28d-ee21ad04dfaa None None] MFA Rules not processed for user `314d887dfdc54571894cae67135d021a`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:52.571339 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-1ad458b7-53ff-495e-b28d-ee21ad04dfaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:52.646369 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7dbe26-ec9a-4352-9e7e-98d424cddd62 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:52.649246 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-1d7dbe26-ec9a-4352-9e7e-98d424cddd62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:52.704627 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1148/2291] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:52 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:52.718500 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad458b7-53ff-495e-b28d-ee21ad04dfaa None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:52.720348 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-1ad458b7-53ff-495e-b28d-ee21ad04dfaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:52.730598 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:52.786681 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7dbe26-ec9a-4352-9e7e-98d424cddd62 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:52.796344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27d489-fc87-43d1-b792-5494b0f22e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:52.798003 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27d489-fc87-43d1-b792-5494b0f22e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:52.799094 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27d489-fc87-43d1-b792-5494b0f22e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:52.799094 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27d489-fc87-43d1-b792-5494b0f22e92 admin admin] PATH_INFO: `/v3/users/ba3bbb4262a94b818bf038b987329b9d` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:52.801030 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27d489-fc87-43d1-b792-5494b0f22e92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba3bbb4262a94b818bf038b987329b9d)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:52.804371 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27d489-fc87-43d1-b792-5494b0f22e92 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:52.821025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad458b7-53ff-495e-b28d-ee21ad04dfaa None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:52.827675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af05169-43c7-49a9-8ad0-e925212e614b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:52.828641 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af05169-43c7-49a9-8ad0-e925212e614b service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:52.828641 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af05169-43c7-49a9-8ad0-e925212e614b service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:52.828641 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af05169-43c7-49a9-8ad0-e925212e614b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:52.844546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af05169-43c7-49a9-8ad0-e925212e614b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:52.849062 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af05169-43c7-49a9-8ad0-e925212e614b service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:52.879903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1149/2292] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:52 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:14:52.903630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-1b27d489-fc87-43d1-b792-5494b0f22e92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba3bbb4262a94b818bf038b987329b9d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:52.923441 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-1b27d489-fc87-43d1-b792-5494b0f22e92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba3bbb4262a94b818bf038b987329b9d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:52.950163 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-1b27d489-fc87-43d1-b792-5494b0f22e92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba3bbb4262a94b818bf038b987329b9d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:52.960087 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:52.966029 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-1b27d489-fc87-43d1-b792-5494b0f22e92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba3bbb4262a94b818bf038b987329b9d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:52.971928 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-1b27d489-fc87-43d1-b792-5494b0f22e92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba3bbb4262a94b818bf038b987329b9d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:52.980988 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1144/2293] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:14:52 2022] DELETE /identity/v3/users/ba3bbb4262a94b818bf038b987329b9d => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:52.987189 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167b514-c133-4bd8-9b9f-a478594b7232 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:52.987946 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167b514-c133-4bd8-9b9f-a478594b7232 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:52.987946 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167b514-c133-4bd8-9b9f-a478594b7232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:53.016731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-7167b514-c133-4bd8-9b9f-a478594b7232 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:53.018009 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-7167b514-c133-4bd8-9b9f-a478594b7232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:53.026922 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27d489-fc87-43d1-b792-5494b0f22e92 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:53.029045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-1b27d489-fc87-43d1-b792-5494b0f22e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:53.118122 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1150/2294] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:53.137771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167b514-c133-4bd8-9b9f-a478594b7232 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:53.138016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27d489-fc87-43d1-b792-5494b0f22e92 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:53.138601 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-7167b514-c133-4bd8-9b9f-a478594b7232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:53.143941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521182ba-5354-4a14-bfae-fd2c70a322e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:53.145945 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521182ba-5354-4a14-bfae-fd2c70a322e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:53.146431 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521182ba-5354-4a14-bfae-fd2c70a322e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:53.147751 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521182ba-5354-4a14-bfae-fd2c70a322e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:53.151633 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-521182ba-5354-4a14-bfae-fd2c70a322e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:53.251347 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167b514-c133-4bd8-9b9f-a478594b7232 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:53.258050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ea629-a3e6-4935-884e-99180c7652f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:53.260618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ea629-a3e6-4935-884e-99180c7652f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:53.261390 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ea629-a3e6-4935-884e-99180c7652f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:53.261633 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521182ba-5354-4a14-bfae-fd2c70a322e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:53.262434 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ea629-a3e6-4935-884e-99180c7652f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:53.264487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521182ba-5354-4a14-bfae-fd2c70a322e4 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:53.281975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1145/2295] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:14:53.298927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ea629-a3e6-4935-884e-99180c7652f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:53.301391 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ea629-a3e6-4935-884e-99180c7652f2 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:53.316687 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1151/2296] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:14:53.705887 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521182ba-5354-4a14-bfae-fd2c70a322e4 service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:53.707783 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-521182ba-5354-4a14-bfae-fd2c70a322e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:53.723956 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521182ba-5354-4a14-bfae-fd2c70a322e4 service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:53.729576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d459dca-7a24-4f21-8b65-a8ae9b688cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:53.730432 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d459dca-7a24-4f21-8b65-a8ae9b688cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:53.730811 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d459dca-7a24-4f21-8b65-a8ae9b688cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:53.731220 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d459dca-7a24-4f21-8b65-a8ae9b688cb9 admin admin] PATH_INFO: `/v3/users/06fbbccaf1e84303abc8bd04cba805bc` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:53.732882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d459dca-7a24-4f21-8b65-a8ae9b688cb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06fbbccaf1e84303abc8bd04cba805bc)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:53.735678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d459dca-7a24-4f21-8b65-a8ae9b688cb9 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:53.819079 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-1d459dca-7a24-4f21-8b65-a8ae9b688cb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06fbbccaf1e84303abc8bd04cba805bc'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:53.837048 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-1d459dca-7a24-4f21-8b65-a8ae9b688cb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06fbbccaf1e84303abc8bd04cba805bc'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:53.865138 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-1d459dca-7a24-4f21-8b65-a8ae9b688cb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06fbbccaf1e84303abc8bd04cba805bc'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:53.889667 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-1d459dca-7a24-4f21-8b65-a8ae9b688cb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06fbbccaf1e84303abc8bd04cba805bc'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:53.891051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-1d459dca-7a24-4f21-8b65-a8ae9b688cb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06fbbccaf1e84303abc8bd04cba805bc'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:53.908305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1146/2297] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:14:53 2022] DELETE /identity/v3/users/06fbbccaf1e84303abc8bd04cba805bc => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:14:53.925369 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b6da4-29bf-4472-a37a-15f7b343d9a1 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:53.925369 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b6da4-29bf-4472-a37a-15f7b343d9a1 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:53.925682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b6da4-29bf-4472-a37a-15f7b343d9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:53.955519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d459dca-7a24-4f21-8b65-a8ae9b688cb9 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:53.958345 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-1d459dca-7a24-4f21-8b65-a8ae9b688cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:53.988683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-347b6da4-29bf-4472-a37a-15f7b343d9a1 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:53.992128 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-347b6da4-29bf-4472-a37a-15f7b343d9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:54.084533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:54.138435 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:54.161578 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d459dca-7a24-4f21-8b65-a8ae9b688cb9 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:54.166955 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1152/2298] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:54.167362 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893068b6-392a-4f25-a4e7-3613cfafbba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:54.168005 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893068b6-392a-4f25-a4e7-3613cfafbba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:54.168278 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893068b6-392a-4f25-a4e7-3613cfafbba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:54.168548 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893068b6-392a-4f25-a4e7-3613cfafbba7 admin admin] PATH_INFO: `/v3/projects/437cf4024a9640bdbe14e3788aaa3ce7` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:54.169865 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893068b6-392a-4f25-a4e7-3613cfafbba7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=437cf4024a9640bdbe14e3788aaa3ce7)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:54.173480 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893068b6-392a-4f25-a4e7-3613cfafbba7 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:54.176817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b6da4-29bf-4472-a37a-15f7b343d9a1 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:54.182420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-347b6da4-29bf-4472-a37a-15f7b343d9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:54.237587 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-893068b6-392a-4f25-a4e7-3613cfafbba7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '437cf4024a9640bdbe14e3788aaa3ce7', 'request_id': 'req-893068b6-392a-4f25-a4e7-3613cfafbba7'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:54.248106 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-893068b6-392a-4f25-a4e7-3613cfafbba7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '437cf4024a9640bdbe14e3788aaa3ce7', 'request_id': 'req-893068b6-392a-4f25-a4e7-3613cfafbba7'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:54.258526 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-893068b6-392a-4f25-a4e7-3613cfafbba7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '437cf4024a9640bdbe14e3788aaa3ce7', 'request_id': 'req-893068b6-392a-4f25-a4e7-3613cfafbba7'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:54.264409 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b6da4-29bf-4472-a37a-15f7b343d9a1 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:54.277600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-893068b6-392a-4f25-a4e7-3613cfafbba7 admin admin] The token cache is being invalidate because project 437cf4024a9640bdbe14e3788aaa3ce7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:14:54.277930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-893068b6-392a-4f25-a4e7-3613cfafbba7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:54.278572 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-893068b6-392a-4f25-a4e7-3613cfafbba7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:54.281008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1147/2299] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:14:53 2022] DELETE /identity/v3/projects/437cf4024a9640bdbe14e3788aaa3ce7 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:54.287429 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab4f56-bea7-47f1-81cb-51f23e7b3849 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:54.287914 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab4f56-bea7-47f1-81cb-51f23e7b3849 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:54.288120 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab4f56-bea7-47f1-81cb-51f23e7b3849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:54.312347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-5aab4f56-bea7-47f1-81cb-51f23e7b3849 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:54.312347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-5aab4f56-bea7-47f1-81cb-51f23e7b3849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:54.333740 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfffac0-9a53-429c-bf72-9347c4468301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:54.334361 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfffac0-9a53-429c-bf72-9347c4468301 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:54.334619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfffac0-9a53-429c-bf72-9347c4468301 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:54.334894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfffac0-9a53-429c-bf72-9347c4468301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:54.336846 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-acfffac0-9a53-429c-bf72-9347c4468301 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:54.411050 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1148/2300] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:14:54.422134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6abb0-78fa-4348-9b38-1e98c79c70db None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:54.422587 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6abb0-78fa-4348-9b38-1e98c79c70db None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:54.423062 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6abb0-78fa-4348-9b38-1e98c79c70db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:54.455673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-46f6abb0-78fa-4348-9b38-1e98c79c70db None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:54.457584 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-46f6abb0-78fa-4348-9b38-1e98c79c70db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:54.470940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1149/2301] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:14:54.478214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6abb0-78fa-4348-9b38-1e98c79c70db None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:54.482242 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfffac0-9a53-429c-bf72-9347c4468301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:54.485036 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfffac0-9a53-429c-bf72-9347c4468301 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:54.497020 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6abb0-78fa-4348-9b38-1e98c79c70db None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:54.504582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1153/2302] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:14:54.507334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95bd8ca-68d5-488c-8999-5dc022a5c501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:54.508436 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bd8ca-68d5-488c-8999-5dc022a5c501 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:54.508879 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bd8ca-68d5-488c-8999-5dc022a5c501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:54.509365 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bd8ca-68d5-488c-8999-5dc022a5c501 admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:54.510794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95bd8ca-68d5-488c-8999-5dc022a5c501 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:14:54.511333 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95bd8ca-68d5-488c-8999-5dc022a5c501 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:54.515050 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95bd8ca-68d5-488c-8999-5dc022a5c501 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:54.540769 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1150/2303] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:14:54.542270 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfffac0-9a53-429c-bf72-9347c4468301 service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:54.554257 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfffac0-9a53-429c-bf72-9347c4468301 service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:54.561453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221dd30-2759-4e47-a9d4-23b008011bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:54.562798 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221dd30-2759-4e47-a9d4-23b008011bbc admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:54.563151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221dd30-2759-4e47-a9d4-23b008011bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:54.563489 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221dd30-2759-4e47-a9d4-23b008011bbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:54.564806 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221dd30-2759-4e47-a9d4-23b008011bbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:54.566913 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221dd30-2759-4e47-a9d4-23b008011bbc admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:54.654501 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1154/2304] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:14:54 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:14:54.664899 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95bd8ca-68d5-488c-8999-5dc022a5c501 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:54.710326 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95bd8ca-68d5-488c-8999-5dc022a5c501 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:54.806377 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:54.883778 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9ee0d-e703-4537-9d73-32ab3ae4515f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:54.884410 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9ee0d-e703-4537-9d73-32ab3ae4515f admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:54.884693 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9ee0d-e703-4537-9d73-32ab3ae4515f admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:54.885069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9ee0d-e703-4537-9d73-32ab3ae4515f admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:54.886897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9ee0d-e703-4537-9d73-32ab3ae4515f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:54.889371 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9ee0d-e703-4537-9d73-32ab3ae4515f admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:54.940574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221dd30-2759-4e47-a9d4-23b008011bbc admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:54.951585 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1151/2305] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:14:54 2022] POST /identity/v3/users => generated 286 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:14:54.957041 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221dd30-2759-4e47-a9d4-23b008011bbc admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:54.964964 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3fc99a-9e99-44a8-b047-6ddbfd83dc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:54.966372 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3fc99a-9e99-44a8-b047-6ddbfd83dc64 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:54.966828 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3fc99a-9e99-44a8-b047-6ddbfd83dc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:54.967882 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3fc99a-9e99-44a8-b047-6ddbfd83dc64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:54.969306 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3fc99a-9e99-44a8-b047-6ddbfd83dc64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:54.974016 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3fc99a-9e99-44a8-b047-6ddbfd83dc64 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:55.003299 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1155/2306] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:55.005169 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9ee0d-e703-4537-9d73-32ab3ae4515f admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:55.019087 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9ee0d-e703-4537-9d73-32ab3ae4515f admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:55.044126 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361a6d7-ef80-4f62-881e-9ecb78334fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:55.045032 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361a6d7-ef80-4f62-881e-9ecb78334fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:55.046157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361a6d7-ef80-4f62-881e-9ecb78334fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:55.046838 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361a6d7-ef80-4f62-881e-9ecb78334fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:55.047890 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361a6d7-ef80-4f62-881e-9ecb78334fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:55.050010 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361a6d7-ef80-4f62-881e-9ecb78334fc1 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:55.124273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1152/2307] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:55.126247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3fc99a-9e99-44a8-b047-6ddbfd83dc64 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:55.157692 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3fc99a-9e99-44a8-b047-6ddbfd83dc64 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:55.185539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645fbe8c-54da-494c-8eb3-205682f6394d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:55.187944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fbe8c-54da-494c-8eb3-205682f6394d admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:55.188481 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fbe8c-54da-494c-8eb3-205682f6394d admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:55.189442 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fbe8c-54da-494c-8eb3-205682f6394d admin admin] PATH_INFO: `/v3/projects/1d0bc6ba67fb47ddb3948d08e25a137e/users/00382f6bad10448ebc1d553c6220f1c7/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:55.218761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361a6d7-ef80-4f62-881e-9ecb78334fc1 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:55.222357 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-2361a6d7-ef80-4f62-881e-9ecb78334fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:55.252796 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645fbe8c-54da-494c-8eb3-205682f6394d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d0bc6ba67fb47ddb3948d08e25a137e, user_id=00382f6bad10448ebc1d553c6220f1c7, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:55.259831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645fbe8c-54da-494c-8eb3-205682f6394d admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:55.267515 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361a6d7-ef80-4f62-881e-9ecb78334fc1 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:55.280707 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:55.299219 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1156/2308] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:14:55 2022] PUT /identity/v3/projects/1d0bc6ba67fb47ddb3948d08e25a137e/users/00382f6bad10448ebc1d553c6220f1c7/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:55.316275 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb110d-586a-4952-886f-43f750523e2b None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:55.316684 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb110d-586a-4952-886f-43f750523e2b None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:55.317128 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb110d-586a-4952-886f-43f750523e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:55.380337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-58fb110d-586a-4952-886f-43f750523e2b None None] MFA Rules not processed for user `00382f6bad10448ebc1d553c6220f1c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:55.381644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-58fb110d-586a-4952-886f-43f750523e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:55.415300 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d934e-0e91-40a0-90d4-e549f9c858dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:55.417338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d934e-0e91-40a0-90d4-e549f9c858dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:55.417930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d934e-0e91-40a0-90d4-e549f9c858dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:55.418201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d934e-0e91-40a0-90d4-e549f9c858dd admin admin] PATH_INFO: `/v3/projects/5359d4a4cbfa49b4b2e58d373de21ab8` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:55.422385 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d934e-0e91-40a0-90d4-e549f9c858dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5359d4a4cbfa49b4b2e58d373de21ab8)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:55.424975 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d934e-0e91-40a0-90d4-e549f9c858dd admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:55.520840 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1157/2309] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:55.552246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb110d-586a-4952-886f-43f750523e2b None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:55.555715 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-58fb110d-586a-4952-886f-43f750523e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:55.560524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-534d934e-0e91-40a0-90d4-e549f9c858dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5359d4a4cbfa49b4b2e58d373de21ab8', 'request_id': 'req-534d934e-0e91-40a0-90d4-e549f9c858dd'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:55.575730 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-534d934e-0e91-40a0-90d4-e549f9c858dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5359d4a4cbfa49b4b2e58d373de21ab8', 'request_id': 'req-534d934e-0e91-40a0-90d4-e549f9c858dd'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:55.589347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-534d934e-0e91-40a0-90d4-e549f9c858dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5359d4a4cbfa49b4b2e58d373de21ab8', 'request_id': 'req-534d934e-0e91-40a0-90d4-e549f9c858dd'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:55.612953 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-534d934e-0e91-40a0-90d4-e549f9c858dd admin admin] The token cache is being invalidate because project 5359d4a4cbfa49b4b2e58d373de21ab8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:14:55.613309 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-534d934e-0e91-40a0-90d4-e549f9c858dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:55.614630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-534d934e-0e91-40a0-90d4-e549f9c858dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:55.617020 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1153/2310] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:14:55 2022] DELETE /identity/v3/projects/5359d4a4cbfa49b4b2e58d373de21ab8 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:55.621222 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d934e-0e91-40a0-90d4-e549f9c858dd admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:55.623096 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-534d934e-0e91-40a0-90d4-e549f9c858dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:55.674594 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb110d-586a-4952-886f-43f750523e2b None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:55.691401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:55.726852 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d934e-0e91-40a0-90d4-e549f9c858dd admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:55.741548 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9074d5-398c-4cc9-8104-f5e86beb3bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:55.742369 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9074d5-398c-4cc9-8104-f5e86beb3bbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:55.742649 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9074d5-398c-4cc9-8104-f5e86beb3bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:55.743003 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9074d5-398c-4cc9-8104-f5e86beb3bbd admin admin] PATH_INFO: `/v3/users/314d887dfdc54571894cae67135d021a` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:55.744664 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9074d5-398c-4cc9-8104-f5e86beb3bbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=314d887dfdc54571894cae67135d021a)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:55.747215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9074d5-398c-4cc9-8104-f5e86beb3bbd admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:55.778919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d6961-ecdc-4f6f-acb1-6d2105dcbd02 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b64b4b172467692d02851b2955b81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:55.781252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d6961-ecdc-4f6f-acb1-6d2105dcbd02 service glance] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:55.781252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d6961-ecdc-4f6f-acb1-6d2105dcbd02 service glance] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:55.781528 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d6961-ecdc-4f6f-acb1-6d2105dcbd02 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:55.785382 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-311d6961-ecdc-4f6f-acb1-6d2105dcbd02 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:55.851700 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9e9074d5-398c-4cc9-8104-f5e86beb3bbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '314d887dfdc54571894cae67135d021a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:55.865016 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9e9074d5-398c-4cc9-8104-f5e86beb3bbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '314d887dfdc54571894cae67135d021a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:55.891622 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9e9074d5-398c-4cc9-8104-f5e86beb3bbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '314d887dfdc54571894cae67135d021a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:55.901817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9e9074d5-398c-4cc9-8104-f5e86beb3bbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '314d887dfdc54571894cae67135d021a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:55.902847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9e9074d5-398c-4cc9-8104-f5e86beb3bbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '314d887dfdc54571894cae67135d021a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:55.912292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1154/2311] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:14:55 2022] DELETE /identity/v3/users/314d887dfdc54571894cae67135d021a => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:14:55.923644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d6961-ecdc-4f6f-acb1-6d2105dcbd02 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:55.925440 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d6961-ecdc-4f6f-acb1-6d2105dcbd02 service glance] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:55.930779 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:55.939119 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1158/2312] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:14:56.374329 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9074d5-398c-4cc9-8104-f5e86beb3bbd admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:56.376776 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9e9074d5-398c-4cc9-8104-f5e86beb3bbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:56.507301 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9074d5-398c-4cc9-8104-f5e86beb3bbd admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:56.513354 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb1dfe-7565-4027-97ad-8d1d18f08a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:56.515220 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb1dfe-7565-4027-97ad-8d1d18f08a9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:56.515687 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb1dfe-7565-4027-97ad-8d1d18f08a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:56.516085 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb1dfe-7565-4027-97ad-8d1d18f08a9d admin admin] PATH_INFO: `/v3/projects/00777eb7189e48368b572b088cdd77b7` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:56.518098 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb1dfe-7565-4027-97ad-8d1d18f08a9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=00777eb7189e48368b572b088cdd77b7)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:56.520743 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb1dfe-7565-4027-97ad-8d1d18f08a9d admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:56.626491 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-05cb1dfe-7565-4027-97ad-8d1d18f08a9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00777eb7189e48368b572b088cdd77b7', 'request_id': 'req-05cb1dfe-7565-4027-97ad-8d1d18f08a9d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:56.645272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-05cb1dfe-7565-4027-97ad-8d1d18f08a9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00777eb7189e48368b572b088cdd77b7', 'request_id': 'req-05cb1dfe-7565-4027-97ad-8d1d18f08a9d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:56.661198 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-05cb1dfe-7565-4027-97ad-8d1d18f08a9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00777eb7189e48368b572b088cdd77b7', 'request_id': 'req-05cb1dfe-7565-4027-97ad-8d1d18f08a9d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:56.665054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:56.716224 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-05cb1dfe-7565-4027-97ad-8d1d18f08a9d admin admin] The token cache is being invalidate because project 00777eb7189e48368b572b088cdd77b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:14:56.716569 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-05cb1dfe-7565-4027-97ad-8d1d18f08a9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:56.723662 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-05cb1dfe-7565-4027-97ad-8d1d18f08a9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:56.725612 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1155/2313] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:14:56 2022] DELETE /identity/v3/projects/00777eb7189e48368b572b088cdd77b7 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:56.734249 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec3b68-08bf-4e89-901e-9e693fcee41f None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:56.736453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec3b68-08bf-4e89-901e-9e693fcee41f None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:56.737575 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec3b68-08bf-4e89-901e-9e693fcee41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:56.812482 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-75ec3b68-08bf-4e89-901e-9e693fcee41f None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:56.814619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-75ec3b68-08bf-4e89-901e-9e693fcee41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:56.977130 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1159/2314] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:56.990239 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6edfc-50e5-4e0f-9274-dfd42d6f89fb None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:56.992042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6edfc-50e5-4e0f-9274-dfd42d6f89fb None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:56.992794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6edfc-50e5-4e0f-9274-dfd42d6f89fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:57.028165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-66e6edfc-50e5-4e0f-9274-dfd42d6f89fb None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:57.030738 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-66e6edfc-50e5-4e0f-9274-dfd42d6f89fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:57.049468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1156/2315] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:14:57.051427 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec3b68-08bf-4e89-901e-9e693fcee41f None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:57.066071 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec3b68-08bf-4e89-901e-9e693fcee41f None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:57.074867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb73aaa-475e-4726-9cbb-597ff764c390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:57.075880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb73aaa-475e-4726-9cbb-597ff764c390 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:57.076886 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb73aaa-475e-4726-9cbb-597ff764c390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:57.077322 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb73aaa-475e-4726-9cbb-597ff764c390 admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:57.078662 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb73aaa-475e-4726-9cbb-597ff764c390 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:14:57.079180 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb73aaa-475e-4726-9cbb-597ff764c390 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:57.082580 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb73aaa-475e-4726-9cbb-597ff764c390 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:57.117491 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1160/2316] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:14:57.119615 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e6edfc-50e5-4e0f-9274-dfd42d6f89fb None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:57.150592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e6edfc-50e5-4e0f-9274-dfd42d6f89fb None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:57.167410 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d20b7-17a4-4cb2-a443-2956ba72d0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:57.168454 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d20b7-17a4-4cb2-a443-2956ba72d0e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:57.168970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d20b7-17a4-4cb2-a443-2956ba72d0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:57.169746 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d20b7-17a4-4cb2-a443-2956ba72d0e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:57.171364 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d20b7-17a4-4cb2-a443-2956ba72d0e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:57.175083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d20b7-17a4-4cb2-a443-2956ba72d0e9 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:57.243562 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1157/2317] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:14:57 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:14:57.250621 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb73aaa-475e-4726-9cbb-597ff764c390 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:57.282206 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb73aaa-475e-4726-9cbb-597ff764c390 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:57.415261 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bf185-57a4-4410-b50a-2f178cf2e652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:57.417629 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bf185-57a4-4410-b50a-2f178cf2e652 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:57.418159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bf185-57a4-4410-b50a-2f178cf2e652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:57.419901 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bf185-57a4-4410-b50a-2f178cf2e652 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:57.419901 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73bf185-57a4-4410-b50a-2f178cf2e652 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:57.423972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73bf185-57a4-4410-b50a-2f178cf2e652 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:57.462934 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1161/2318] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:14:57 2022] POST /identity/v3/users => generated 292 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:14:57.471097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d20b7-17a4-4cb2-a443-2956ba72d0e9 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:57.520768 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d20b7-17a4-4cb2-a443-2956ba72d0e9 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:57.544150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c1a7b-fbe4-4c44-95fa-c79968846bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:57.546681 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c1a7b-fbe4-4c44-95fa-c79968846bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:57.547552 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c1a7b-fbe4-4c44-95fa-c79968846bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:57.548287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c1a7b-fbe4-4c44-95fa-c79968846bc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:57.552655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c1a7b-fbe4-4c44-95fa-c79968846bc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:57.555991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c1a7b-fbe4-4c44-95fa-c79968846bc7 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:57.606070 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1158/2319] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:14:57.613853 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bf185-57a4-4410-b50a-2f178cf2e652 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:57.634106 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bf185-57a4-4410-b50a-2f178cf2e652 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:57.641600 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85481a1a-00ef-4275-905f-38cffe1febbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:57.643282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85481a1a-00ef-4275-905f-38cffe1febbf admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:57.643973 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85481a1a-00ef-4275-905f-38cffe1febbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:57.644493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85481a1a-00ef-4275-905f-38cffe1febbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:57.647145 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85481a1a-00ef-4275-905f-38cffe1febbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:57.651820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85481a1a-00ef-4275-905f-38cffe1febbf admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:57.673982 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1162/2320] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:14:57.675465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c1a7b-fbe4-4c44-95fa-c79968846bc7 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:57.687934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c1a7b-fbe4-4c44-95fa-c79968846bc7 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:57.696925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c77ee-fe04-4045-869f-2bcbe8fe0231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:57.697641 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c77ee-fe04-4045-869f-2bcbe8fe0231 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:57.698148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c77ee-fe04-4045-869f-2bcbe8fe0231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:57.698656 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c77ee-fe04-4045-869f-2bcbe8fe0231 admin admin] PATH_INFO: `/v3/projects/2df95fced5db45febdb970f38529d1c7/users/b4ea3e71d9524328afae80d971a2c9b6/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:57.730062 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c77ee-fe04-4045-869f-2bcbe8fe0231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2df95fced5db45febdb970f38529d1c7, user_id=b4ea3e71d9524328afae80d971a2c9b6, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:57.732976 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c77ee-fe04-4045-869f-2bcbe8fe0231 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:57.769424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1159/2321] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:14:57 2022] PUT /identity/v3/projects/2df95fced5db45febdb970f38529d1c7/users/b4ea3e71d9524328afae80d971a2c9b6/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:14:57.782331 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a18b7a-0684-4cf4-bd04-6e9efcf93d5a None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:57.782958 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a18b7a-0684-4cf4-bd04-6e9efcf93d5a None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:57.782958 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a18b7a-0684-4cf4-bd04-6e9efcf93d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:57.899845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-79a18b7a-0684-4cf4-bd04-6e9efcf93d5a None None] MFA Rules not processed for user `b4ea3e71d9524328afae80d971a2c9b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:57.903453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-79a18b7a-0684-4cf4-bd04-6e9efcf93d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:58.071888 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:58.111335 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1163/2322] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:14:58.148408 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c77ee-fe04-4045-869f-2bcbe8fe0231 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:58.152782 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-789c77ee-fe04-4045-869f-2bcbe8fe0231 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:58.271091 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c77ee-fe04-4045-869f-2bcbe8fe0231 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:58.280884 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff8c744-949b-47fa-ad98-230653fe4a02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:58.282171 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8c744-949b-47fa-ad98-230653fe4a02 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:58.283039 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8c744-949b-47fa-ad98-230653fe4a02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:58.283525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8c744-949b-47fa-ad98-230653fe4a02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:58.312760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff8c744-949b-47fa-ad98-230653fe4a02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:58.316509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff8c744-949b-47fa-ad98-230653fe4a02 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:58.337500 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1160/2323] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:14:58.375818 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a18b7a-0684-4cf4-bd04-6e9efcf93d5a None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:58.561482 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a18b7a-0684-4cf4-bd04-6e9efcf93d5a None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:58.581237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbb379-f7b0-4496-878a-ae8b0c6d47e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:58.582783 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbb379-f7b0-4496-878a-ae8b0c6d47e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:58.583510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbb379-f7b0-4496-878a-ae8b0c6d47e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:58.583976 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbb379-f7b0-4496-878a-ae8b0c6d47e9 admin admin] PATH_INFO: `/v3/users/b4ea3e71d9524328afae80d971a2c9b6` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:58.587256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbb379-f7b0-4496-878a-ae8b0c6d47e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4ea3e71d9524328afae80d971a2c9b6)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:58.591179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbb379-f7b0-4496-878a-ae8b0c6d47e9 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:58.739440 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-ddbbb379-f7b0-4496-878a-ae8b0c6d47e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4ea3e71d9524328afae80d971a2c9b6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:58.786915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-ddbbb379-f7b0-4496-878a-ae8b0c6d47e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4ea3e71d9524328afae80d971a2c9b6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:58.823059 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-ddbbb379-f7b0-4496-878a-ae8b0c6d47e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4ea3e71d9524328afae80d971a2c9b6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:58.840132 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-ddbbb379-f7b0-4496-878a-ae8b0c6d47e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4ea3e71d9524328afae80d971a2c9b6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:58.841966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-ddbbb379-f7b0-4496-878a-ae8b0c6d47e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4ea3e71d9524328afae80d971a2c9b6'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:14:58.860512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1164/2324] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:14:58 2022] DELETE /identity/v3/users/b4ea3e71d9524328afae80d971a2c9b6 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:14:58.866887 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd2a33-1431-4bd1-a051-a8f196942c04 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:58.867670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd2a33-1431-4bd1-a051-a8f196942c04 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:58.867992 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd2a33-1431-4bd1-a051-a8f196942c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:58.901332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-3abd2a33-1431-4bd1-a051-a8f196942c04 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:58.903486 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-3abd2a33-1431-4bd1-a051-a8f196942c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:59.062135 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1161/2325] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:14:59.093405 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbb379-f7b0-4496-878a-ae8b0c6d47e9 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:14:59.096041 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ddbbb379-f7b0-4496-878a-ae8b0c6d47e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:59.223806 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbb379-f7b0-4496-878a-ae8b0c6d47e9 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:14:59.231597 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc1d2c-ec06-4dad-a9e9-4c05f09c6716 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:14:59.233363 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc1d2c-ec06-4dad-a9e9-4c05f09c6716 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:59.233977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc1d2c-ec06-4dad-a9e9-4c05f09c6716 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:59.234362 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc1d2c-ec06-4dad-a9e9-4c05f09c6716 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:59.258656 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc1d2c-ec06-4dad-a9e9-4c05f09c6716 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:14:59.264001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc1d2c-ec06-4dad-a9e9-4c05f09c6716 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:14:59.292137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1165/2326] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:14:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:14:59.689089 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367148f-865d-4467-98c2-2aab2cc92192 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:14:59.689889 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367148f-865d-4467-98c2-2aab2cc92192 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:14:59.690530 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367148f-865d-4467-98c2-2aab2cc92192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:14:59.735344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-1367148f-865d-4467-98c2-2aab2cc92192 None None] MFA Rules not processed for user `e75b537d44524de192282b10825293f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:14:59.740464 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-1367148f-865d-4467-98c2-2aab2cc92192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:14:59.817199 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:14:59.897791 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1162/2327] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:14:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:15:00.285882 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc1d2c-ec06-4dad-a9e9-4c05f09c6716 service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:00.288701 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-61dc1d2c-ec06-4dad-a9e9-4c05f09c6716 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:00.317547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc1d2c-ec06-4dad-a9e9-4c05f09c6716 service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:00.332239 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e854300-687c-40df-a955-fe83e12d46d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:00.333400 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e854300-687c-40df-a955-fe83e12d46d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:00.333897 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e854300-687c-40df-a955-fe83e12d46d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:00.334487 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e854300-687c-40df-a955-fe83e12d46d7 admin admin] PATH_INFO: `/v3/projects/2df95fced5db45febdb970f38529d1c7` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:00.337031 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e854300-687c-40df-a955-fe83e12d46d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2df95fced5db45febdb970f38529d1c7)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:00.340169 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e854300-687c-40df-a955-fe83e12d46d7 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:00.363475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:15:00.455181 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8e854300-687c-40df-a955-fe83e12d46d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2df95fced5db45febdb970f38529d1c7', 'request_id': 'req-8e854300-687c-40df-a955-fe83e12d46d7'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:00.466292 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8e854300-687c-40df-a955-fe83e12d46d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2df95fced5db45febdb970f38529d1c7', 'request_id': 'req-8e854300-687c-40df-a955-fe83e12d46d7'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:00.485758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8e854300-687c-40df-a955-fe83e12d46d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2df95fced5db45febdb970f38529d1c7', 'request_id': 'req-8e854300-687c-40df-a955-fe83e12d46d7'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:00.518572 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8e854300-687c-40df-a955-fe83e12d46d7 admin admin] The token cache is being invalidate because project 2df95fced5db45febdb970f38529d1c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:15:00.519682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8e854300-687c-40df-a955-fe83e12d46d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:00.520906 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8e854300-687c-40df-a955-fe83e12d46d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:00.523714 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1166/2328] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:15:00 2022] DELETE /identity/v3/projects/2df95fced5db45febdb970f38529d1c7 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:15:00.531731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc29306-297e-4e99-9bff-989d419a2569 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:00.532171 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc29306-297e-4e99-9bff-989d419a2569 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:00.532633 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc29306-297e-4e99-9bff-989d419a2569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:00.569927 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-ddc29306-297e-4e99-9bff-989d419a2569 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:15:00.573192 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-ddc29306-297e-4e99-9bff-989d419a2569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:00.679286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1163/2329] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:15:00.691871 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb057b2-01df-453b-bbd2-fab89335ad4e None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:00.692556 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb057b2-01df-453b-bbd2-fab89335ad4e None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:00.693056 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb057b2-01df-453b-bbd2-fab89335ad4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:00.730250 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-9eb057b2-01df-453b-bbd2-fab89335ad4e None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:15:00.732166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-9eb057b2-01df-453b-bbd2-fab89335ad4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:00.746845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1167/2330] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:15:00.753780 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc29306-297e-4e99-9bff-989d419a2569 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:00.782878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc29306-297e-4e99-9bff-989d419a2569 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:00.790108 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6b2dd-5207-4470-b703-7c13f455eeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:00.791047 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6b2dd-5207-4470-b703-7c13f455eeb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:00.791475 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6b2dd-5207-4470-b703-7c13f455eeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:00.791979 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6b2dd-5207-4470-b703-7c13f455eeb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:00.792897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6b2dd-5207-4470-b703-7c13f455eeb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:15:00.793352 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6b2dd-5207-4470-b703-7c13f455eeb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:00.796479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6b2dd-5207-4470-b703-7c13f455eeb0 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:00.818981 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1164/2331] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:15:00.825933 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb057b2-01df-453b-bbd2-fab89335ad4e None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:00.849666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb057b2-01df-453b-bbd2-fab89335ad4e None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:00.858233 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f5ed4-a867-45f2-8b76-30c6185d3467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:00.859264 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f5ed4-a867-45f2-8b76-30c6185d3467 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:00.859695 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f5ed4-a867-45f2-8b76-30c6185d3467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:00.860147 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f5ed4-a867-45f2-8b76-30c6185d3467 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:00.861222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f5ed4-a867-45f2-8b76-30c6185d3467 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:00.865752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f5ed4-a867-45f2-8b76-30c6185d3467 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:00.928120 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1168/2332] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:15:00 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:15:00.935418 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6b2dd-5207-4470-b703-7c13f455eeb0 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:00.951884 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6b2dd-5207-4470-b703-7c13f455eeb0 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:01.078937 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c1ab5-b86f-475d-b940-f4a5308e8f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:01.079902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c1ab5-b86f-475d-b940-f4a5308e8f81 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:01.080299 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c1ab5-b86f-475d-b940-f4a5308e8f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:01.080865 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c1ab5-b86f-475d-b940-f4a5308e8f81 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:01.082338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c1ab5-b86f-475d-b940-f4a5308e8f81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:01.088209 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c1ab5-b86f-475d-b940-f4a5308e8f81 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:01.128202 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1165/2333] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:15:00 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:15:01.134885 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f5ed4-a867-45f2-8b76-30c6185d3467 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:01.158354 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f5ed4-a867-45f2-8b76-30c6185d3467 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:01.174618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd47b9-4a1a-4317-b999-80fe77c50739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:01.177344 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd47b9-4a1a-4317-b999-80fe77c50739 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:01.178134 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd47b9-4a1a-4317-b999-80fe77c50739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:01.178837 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd47b9-4a1a-4317-b999-80fe77c50739 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:01.180704 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd47b9-4a1a-4317-b999-80fe77c50739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:01.184760 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd47b9-4a1a-4317-b999-80fe77c50739 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:01.210731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1169/2334] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:15:01.211806 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c1ab5-b86f-475d-b940-f4a5308e8f81 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:01.230642 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c1ab5-b86f-475d-b940-f4a5308e8f81 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:01.241940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf433178-6c44-4c43-bf71-0f4a7ca27421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:01.243528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf433178-6c44-4c43-bf71-0f4a7ca27421 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:01.244234 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf433178-6c44-4c43-bf71-0f4a7ca27421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:01.245330 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf433178-6c44-4c43-bf71-0f4a7ca27421 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:01.246750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf433178-6c44-4c43-bf71-0f4a7ca27421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:01.250840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf433178-6c44-4c43-bf71-0f4a7ca27421 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:01.274343 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1166/2335] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:15:01.283313 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd47b9-4a1a-4317-b999-80fe77c50739 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:01.301224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd47b9-4a1a-4317-b999-80fe77c50739 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:01.306655 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e1762-2a20-4168-80b6-7d6224e3d9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:01.307602 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e1762-2a20-4168-80b6-7d6224e3d9a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:01.308054 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e1762-2a20-4168-80b6-7d6224e3d9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:01.308501 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e1762-2a20-4168-80b6-7d6224e3d9a4 admin admin] PATH_INFO: `/v3/projects/3476c2de77584f69bc25100ac93e9ed7/users/52e5fff32fa3435cb4609e8e185b7e91/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:01.342275 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e1762-2a20-4168-80b6-7d6224e3d9a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3476c2de77584f69bc25100ac93e9ed7, user_id=52e5fff32fa3435cb4609e8e185b7e91, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:01.342275 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e1762-2a20-4168-80b6-7d6224e3d9a4 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:01.366184 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1170/2336] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:15:01 2022] PUT /identity/v3/projects/3476c2de77584f69bc25100ac93e9ed7/users/52e5fff32fa3435cb4609e8e185b7e91/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:15:01.388014 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2c7fb-a42e-43e3-b37f-c49cf0238abc None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:01.388510 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2c7fb-a42e-43e3-b37f-c49cf0238abc None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:01.389070 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2c7fb-a42e-43e3-b37f-c49cf0238abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:01.456706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-91a2c7fb-a42e-43e3-b37f-c49cf0238abc None None] MFA Rules not processed for user `52e5fff32fa3435cb4609e8e185b7e91`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:15:01.458852 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-91a2c7fb-a42e-43e3-b37f-c49cf0238abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:01.527344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:15:01.604591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1167/2337] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:15:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:15:01.628478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e1762-2a20-4168-80b6-7d6224e3d9a4 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:01.632715 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-7e5e1762-2a20-4168-80b6-7d6224e3d9a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:01.855695 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e1762-2a20-4168-80b6-7d6224e3d9a4 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:01.879073 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf8268-3dd3-4161-98a1-92f20b5fc60c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:01.880019 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf8268-3dd3-4161-98a1-92f20b5fc60c service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:01.880455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf8268-3dd3-4161-98a1-92f20b5fc60c service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:01.880930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf8268-3dd3-4161-98a1-92f20b5fc60c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:01.915276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf8268-3dd3-4161-98a1-92f20b5fc60c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:01.917691 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf8268-3dd3-4161-98a1-92f20b5fc60c service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:01.951436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1171/2338] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:15:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:15:02.010630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2c7fb-a42e-43e3-b37f-c49cf0238abc None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:02.243595 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2c7fb-a42e-43e3-b37f-c49cf0238abc None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:02.270669 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac47448-f416-43a8-b226-e209e3d8df51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:02.273523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac47448-f416-43a8-b226-e209e3d8df51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:02.274005 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac47448-f416-43a8-b226-e209e3d8df51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:02.274448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac47448-f416-43a8-b226-e209e3d8df51 admin admin] PATH_INFO: `/v3/users/52e5fff32fa3435cb4609e8e185b7e91` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:02.281292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac47448-f416-43a8-b226-e209e3d8df51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52e5fff32fa3435cb4609e8e185b7e91)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:02.284601 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac47448-f416-43a8-b226-e209e3d8df51 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:02.393066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-aac47448-f416-43a8-b226-e209e3d8df51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52e5fff32fa3435cb4609e8e185b7e91'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:02.409172 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-aac47448-f416-43a8-b226-e209e3d8df51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52e5fff32fa3435cb4609e8e185b7e91'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:02.437562 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-aac47448-f416-43a8-b226-e209e3d8df51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52e5fff32fa3435cb4609e8e185b7e91'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:02.452601 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-aac47448-f416-43a8-b226-e209e3d8df51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52e5fff32fa3435cb4609e8e185b7e91'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:02.454648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-aac47448-f416-43a8-b226-e209e3d8df51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52e5fff32fa3435cb4609e8e185b7e91'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:02.463720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1168/2339] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:15:02 2022] DELETE /identity/v3/users/52e5fff32fa3435cb4609e8e185b7e91 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:15:02.469876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da691903-b8b6-4224-958c-c16e77c624a9 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:02.470667 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da691903-b8b6-4224-958c-c16e77c624a9 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:02.470795 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da691903-b8b6-4224-958c-c16e77c624a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:02.524775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-da691903-b8b6-4224-958c-c16e77c624a9 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:15:02.527110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-da691903-b8b6-4224-958c-c16e77c624a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:02.653643 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:15:02.720972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1172/2340] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:15:02.736230 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac47448-f416-43a8-b226-e209e3d8df51 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:02.738954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-aac47448-f416-43a8-b226-e209e3d8df51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:02.834627 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac47448-f416-43a8-b226-e209e3d8df51 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:02.838562 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c665fa-fc2d-4157-8a1b-f9615496b761 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:02.839187 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c665fa-fc2d-4157-8a1b-f9615496b761 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:02.839467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c665fa-fc2d-4157-8a1b-f9615496b761 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:02.839738 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c665fa-fc2d-4157-8a1b-f9615496b761 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:02.847509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:15:02.854772 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c665fa-fc2d-4157-8a1b-f9615496b761 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:02.857403 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c665fa-fc2d-4157-8a1b-f9615496b761 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:02.877990 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1169/2341] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:15:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:15:03.667790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da691903-b8b6-4224-958c-c16e77c624a9 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:03.671097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-da691903-b8b6-4224-958c-c16e77c624a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:03.700850 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da691903-b8b6-4224-958c-c16e77c624a9 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:03.708674 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ca151-ac54-4489-a607-5aa700ae9205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:03.710147 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ca151-ac54-4489-a607-5aa700ae9205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:03.710891 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ca151-ac54-4489-a607-5aa700ae9205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:03.711612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ca151-ac54-4489-a607-5aa700ae9205 admin admin] PATH_INFO: `/v3/projects/3476c2de77584f69bc25100ac93e9ed7` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:03.719200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ca151-ac54-4489-a607-5aa700ae9205 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3476c2de77584f69bc25100ac93e9ed7)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:03.738383 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ca151-ac54-4489-a607-5aa700ae9205 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:03.883767 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-712ca151-ac54-4489-a607-5aa700ae9205 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3476c2de77584f69bc25100ac93e9ed7', 'request_id': 'req-712ca151-ac54-4489-a607-5aa700ae9205'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:03.895493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-712ca151-ac54-4489-a607-5aa700ae9205 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3476c2de77584f69bc25100ac93e9ed7', 'request_id': 'req-712ca151-ac54-4489-a607-5aa700ae9205'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:03.910245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-712ca151-ac54-4489-a607-5aa700ae9205 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3476c2de77584f69bc25100ac93e9ed7', 'request_id': 'req-712ca151-ac54-4489-a607-5aa700ae9205'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:03.933804 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-712ca151-ac54-4489-a607-5aa700ae9205 admin admin] The token cache is being invalidate because project 3476c2de77584f69bc25100ac93e9ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:15:03.934437 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-712ca151-ac54-4489-a607-5aa700ae9205 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:03.935700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-712ca151-ac54-4489-a607-5aa700ae9205 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:03.939145 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1173/2342] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:15:03 2022] DELETE /identity/v3/projects/3476c2de77584f69bc25100ac93e9ed7 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:15:03.945292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49a875-fd6f-43b5-8350-db6d7bee4981 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:03.946091 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49a875-fd6f-43b5-8350-db6d7bee4981 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:03.946454 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49a875-fd6f-43b5-8350-db6d7bee4981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:03.986406 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-0e49a875-fd6f-43b5-8350-db6d7bee4981 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:15:03.988381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-0e49a875-fd6f-43b5-8350-db6d7bee4981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:04.154389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1170/2343] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:15:04.156960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6805181-4209-41a4-b21b-9beadc2b82cc None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:04.157398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6805181-4209-41a4-b21b-9beadc2b82cc None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:04.157881 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6805181-4209-41a4-b21b-9beadc2b82cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:04.187168 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-a6805181-4209-41a4-b21b-9beadc2b82cc None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:15:04.188706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a6805181-4209-41a4-b21b-9beadc2b82cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:04.203486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1174/2344] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:15:04.210475 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e49a875-fd6f-43b5-8350-db6d7bee4981 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:04.224545 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e49a875-fd6f-43b5-8350-db6d7bee4981 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:04.235245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a5610-c022-40dc-a51a-dc920462eb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:04.235923 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a5610-c022-40dc-a51a-dc920462eb1c admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:04.236263 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a5610-c022-40dc-a51a-dc920462eb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:04.236596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a5610-c022-40dc-a51a-dc920462eb1c admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:04.237565 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a5610-c022-40dc-a51a-dc920462eb1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:15:04.237965 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a5610-c022-40dc-a51a-dc920462eb1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:04.239766 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a5610-c022-40dc-a51a-dc920462eb1c admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:04.254200 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1171/2345] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:15:04.261350 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6805181-4209-41a4-b21b-9beadc2b82cc None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:04.278204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6805181-4209-41a4-b21b-9beadc2b82cc None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:04.284584 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b8362-4951-4504-9c0b-1a6b8ec1dcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:04.285600 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b8362-4951-4504-9c0b-1a6b8ec1dcea admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:04.286053 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b8362-4951-4504-9c0b-1a6b8ec1dcea admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:04.286505 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b8362-4951-4504-9c0b-1a6b8ec1dcea admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:04.287361 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b8362-4951-4504-9c0b-1a6b8ec1dcea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:04.290070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b8362-4951-4504-9c0b-1a6b8ec1dcea admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:04.333358 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1175/2346] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:15:04 2022] POST /identity/v3/projects => generated 375 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:15:04.335570 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a5610-c022-40dc-a51a-dc920462eb1c admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:04.354898 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a5610-c022-40dc-a51a-dc920462eb1c admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:04.457086 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8ffab-daee-4849-aca9-cfb273b3ea9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:04.458090 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8ffab-daee-4849-aca9-cfb273b3ea9d admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:04.458596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8ffab-daee-4849-aca9-cfb273b3ea9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:04.458942 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8ffab-daee-4849-aca9-cfb273b3ea9d admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:04.459932 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8ffab-daee-4849-aca9-cfb273b3ea9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:04.462919 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8ffab-daee-4849-aca9-cfb273b3ea9d admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:04.505756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1172/2347] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:15:04 2022] POST /identity/v3/users => generated 288 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:15:04.510849 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b8362-4951-4504-9c0b-1a6b8ec1dcea admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:04.528230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b8362-4951-4504-9c0b-1a6b8ec1dcea admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:04.536186 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646beda2-9b3a-47ec-ada3-7d67630aabc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:04.537086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646beda2-9b3a-47ec-ada3-7d67630aabc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:04.537521 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646beda2-9b3a-47ec-ada3-7d67630aabc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:04.537979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646beda2-9b3a-47ec-ada3-7d67630aabc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:04.538982 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646beda2-9b3a-47ec-ada3-7d67630aabc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:04.541587 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646beda2-9b3a-47ec-ada3-7d67630aabc7 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:04.559826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1176/2348] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:15:04.565856 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8ffab-daee-4849-aca9-cfb273b3ea9d admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:04.587153 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8ffab-daee-4849-aca9-cfb273b3ea9d admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:04.594087 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c7c04-bb2f-4572-b107-efedd770cb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:04.594993 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c7c04-bb2f-4572-b107-efedd770cb52 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:04.595345 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c7c04-bb2f-4572-b107-efedd770cb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:04.595666 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c7c04-bb2f-4572-b107-efedd770cb52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:04.596315 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c7c04-bb2f-4572-b107-efedd770cb52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:04.597934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c7c04-bb2f-4572-b107-efedd770cb52 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:04.610368 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1173/2349] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:15:04.617284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646beda2-9b3a-47ec-ada3-7d67630aabc7 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:04.632724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646beda2-9b3a-47ec-ada3-7d67630aabc7 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:04.641315 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52474d2-47d1-414f-a127-7c662d1322f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:04.642455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52474d2-47d1-414f-a127-7c662d1322f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:04.642772 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52474d2-47d1-414f-a127-7c662d1322f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:04.643235 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52474d2-47d1-414f-a127-7c662d1322f6 admin admin] PATH_INFO: `/v3/projects/b66d64e1b8fd4197a0fffd7854a063dc/users/22e08cf0913149b09e97dff416aafb86/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:04.672509 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52474d2-47d1-414f-a127-7c662d1322f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b66d64e1b8fd4197a0fffd7854a063dc, user_id=22e08cf0913149b09e97dff416aafb86, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:04.677824 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52474d2-47d1-414f-a127-7c662d1322f6 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:04.715933 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1177/2350] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:15:04 2022] PUT /identity/v3/projects/b66d64e1b8fd4197a0fffd7854a063dc/users/22e08cf0913149b09e97dff416aafb86/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:15:04.734586 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62284842-6833-4dca-9515-5ecc8c5c91b8 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:04.735113 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62284842-6833-4dca-9515-5ecc8c5c91b8 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:04.735630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62284842-6833-4dca-9515-5ecc8c5c91b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:04.788504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-62284842-6833-4dca-9515-5ecc8c5c91b8 None None] MFA Rules not processed for user `22e08cf0913149b09e97dff416aafb86`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:15:04.790645 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-62284842-6833-4dca-9515-5ecc8c5c91b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:04.894344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:15:04.933436 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1174/2351] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:15:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:15:04.946805 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52474d2-47d1-414f-a127-7c662d1322f6 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:04.949184 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f52474d2-47d1-414f-a127-7c662d1322f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:05.047051 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52474d2-47d1-414f-a127-7c662d1322f6 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:05.051736 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6429413-e802-4d08-9085-dcd7fb20b2d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:05.052646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6429413-e802-4d08-9085-dcd7fb20b2d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:05.053093 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6429413-e802-4d08-9085-dcd7fb20b2d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:05.053509 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6429413-e802-4d08-9085-dcd7fb20b2d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:05.067887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6429413-e802-4d08-9085-dcd7fb20b2d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:05.070260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6429413-e802-4d08-9085-dcd7fb20b2d3 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:05.080570 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:15:05.091725 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1178/2352] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:15:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:15:11.684560 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b731ed-8553-4571-8729-bedfb7a57afa None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:11.685444 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b731ed-8553-4571-8729-bedfb7a57afa None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:11.686565 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b731ed-8553-4571-8729-bedfb7a57afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:11.722015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-76b731ed-8553-4571-8729-bedfb7a57afa None None] MFA Rules not processed for user `be17fa7c5ebe49efa4bcae7c721a5e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:15:11.723723 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-76b731ed-8553-4571-8729-bedfb7a57afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:11.834932 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1175/2353] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:15:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:15:18.401252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6429413-e802-4d08-9085-dcd7fb20b2d3 service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:18.510441 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6429413-e802-4d08-9085-dcd7fb20b2d3 service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:18.520083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1a98f-9dac-486f-b7ec-eceadee47552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:18.521604 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1a98f-9dac-486f-b7ec-eceadee47552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:18.521848 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1a98f-9dac-486f-b7ec-eceadee47552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:18.522331 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1a98f-9dac-486f-b7ec-eceadee47552 admin admin] PATH_INFO: `/v3/users/22e08cf0913149b09e97dff416aafb86` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:18.525042 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1a98f-9dac-486f-b7ec-eceadee47552 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22e08cf0913149b09e97dff416aafb86)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:18.529278 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1a98f-9dac-486f-b7ec-eceadee47552 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:18.627407 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-17a1a98f-9dac-486f-b7ec-eceadee47552 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22e08cf0913149b09e97dff416aafb86'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:18.655512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-17a1a98f-9dac-486f-b7ec-eceadee47552 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22e08cf0913149b09e97dff416aafb86'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:18.672445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-17a1a98f-9dac-486f-b7ec-eceadee47552 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22e08cf0913149b09e97dff416aafb86'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:18.697204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-17a1a98f-9dac-486f-b7ec-eceadee47552 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22e08cf0913149b09e97dff416aafb86'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:18.698430 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-17a1a98f-9dac-486f-b7ec-eceadee47552 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22e08cf0913149b09e97dff416aafb86'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:18.712831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1179/2354] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:15:18 2022] DELETE /identity/v3/users/22e08cf0913149b09e97dff416aafb86 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:15:18.714946 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c37ef-a702-4a6b-91d7-1b7be0decc9b None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:18.717100 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c37ef-a702-4a6b-91d7-1b7be0decc9b None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:18.717743 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c37ef-a702-4a6b-91d7-1b7be0decc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:18.752306 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-586c37ef-a702-4a6b-91d7-1b7be0decc9b None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:15:18.754368 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-586c37ef-a702-4a6b-91d7-1b7be0decc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:18.882020 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1176/2355] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:15:18.903800 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1a98f-9dac-486f-b7ec-eceadee47552 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:18.904540 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-17a1a98f-9dac-486f-b7ec-eceadee47552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:19.018680 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1a98f-9dac-486f-b7ec-eceadee47552 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:19.024652 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27857ac7-886d-4827-a9d3-a7660a3d9b40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:19.025596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27857ac7-886d-4827-a9d3-a7660a3d9b40 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:19.026093 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27857ac7-886d-4827-a9d3-a7660a3d9b40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:19.026822 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27857ac7-886d-4827-a9d3-a7660a3d9b40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:19.045286 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27857ac7-886d-4827-a9d3-a7660a3d9b40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:19.049392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27857ac7-886d-4827-a9d3-a7660a3d9b40 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:19.069427 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1180/2356] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:15:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:15:19.527163 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c37ef-a702-4a6b-91d7-1b7be0decc9b None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:19.529403 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-586c37ef-a702-4a6b-91d7-1b7be0decc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:19.552960 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c37ef-a702-4a6b-91d7-1b7be0decc9b None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:19.561149 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b74f60-ac64-4fa8-b7f1-9933a311fc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:19.562886 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b74f60-ac64-4fa8-b7f1-9933a311fc7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:19.563380 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b74f60-ac64-4fa8-b7f1-9933a311fc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:19.563854 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b74f60-ac64-4fa8-b7f1-9933a311fc7d admin admin] PATH_INFO: `/v3/projects/b66d64e1b8fd4197a0fffd7854a063dc` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:19.566032 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b74f60-ac64-4fa8-b7f1-9933a311fc7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b66d64e1b8fd4197a0fffd7854a063dc)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:19.568711 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b74f60-ac64-4fa8-b7f1-9933a311fc7d admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:19.659044 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-34b74f60-ac64-4fa8-b7f1-9933a311fc7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b66d64e1b8fd4197a0fffd7854a063dc', 'request_id': 'req-34b74f60-ac64-4fa8-b7f1-9933a311fc7d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:19.672160 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-34b74f60-ac64-4fa8-b7f1-9933a311fc7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b66d64e1b8fd4197a0fffd7854a063dc', 'request_id': 'req-34b74f60-ac64-4fa8-b7f1-9933a311fc7d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:19.681470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-34b74f60-ac64-4fa8-b7f1-9933a311fc7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b66d64e1b8fd4197a0fffd7854a063dc', 'request_id': 'req-34b74f60-ac64-4fa8-b7f1-9933a311fc7d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:19.699920 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-34b74f60-ac64-4fa8-b7f1-9933a311fc7d admin admin] The token cache is being invalidate because project b66d64e1b8fd4197a0fffd7854a063dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:15:19.700195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-34b74f60-ac64-4fa8-b7f1-9933a311fc7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:19.700888 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-34b74f60-ac64-4fa8-b7f1-9933a311fc7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:19.703158 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1177/2357] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:15:19 2022] DELETE /identity/v3/projects/b66d64e1b8fd4197a0fffd7854a063dc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:15:19.707975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152c4bf-9ed6-41ea-ad93-65064ab8b830 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:19.708852 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152c4bf-9ed6-41ea-ad93-65064ab8b830 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:19.708852 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152c4bf-9ed6-41ea-ad93-65064ab8b830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:19.736432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-e152c4bf-9ed6-41ea-ad93-65064ab8b830 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:15:19.737679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-e152c4bf-9ed6-41ea-ad93-65064ab8b830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:19.759133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:15:19.830589 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1181/2358] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:15:19.846882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63c1cc-5609-4624-8fb2-f2cc2c2dc41d None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:19.847484 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63c1cc-5609-4624-8fb2-f2cc2c2dc41d None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:19.847567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63c1cc-5609-4624-8fb2-f2cc2c2dc41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:19.881926 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-ca63c1cc-5609-4624-8fb2-f2cc2c2dc41d None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:15:19.884544 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-ca63c1cc-5609-4624-8fb2-f2cc2c2dc41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:19.900329 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1178/2359] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:15:19.909903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152c4bf-9ed6-41ea-ad93-65064ab8b830 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:19.925451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152c4bf-9ed6-41ea-ad93-65064ab8b830 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:19.934319 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e0b7f-c6c4-4f4a-82a1-ab44b6fc1117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:19.935098 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e0b7f-c6c4-4f4a-82a1-ab44b6fc1117 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:19.935385 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e0b7f-c6c4-4f4a-82a1-ab44b6fc1117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:19.935769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e0b7f-c6c4-4f4a-82a1-ab44b6fc1117 admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:19.937457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e0b7f-c6c4-4f4a-82a1-ab44b6fc1117 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:15:19.937796 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e0b7f-c6c4-4f4a-82a1-ab44b6fc1117 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:19.939438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e0b7f-c6c4-4f4a-82a1-ab44b6fc1117 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:19.956710 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1182/2360] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:15:19.962670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63c1cc-5609-4624-8fb2-f2cc2c2dc41d None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:19.981489 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63c1cc-5609-4624-8fb2-f2cc2c2dc41d None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:19.990459 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157775e6-5aa9-4a84-9515-359ba0559c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:19.991169 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157775e6-5aa9-4a84-9515-359ba0559c6e admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:19.991450 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157775e6-5aa9-4a84-9515-359ba0559c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:19.991728 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157775e6-5aa9-4a84-9515-359ba0559c6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:19.992398 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157775e6-5aa9-4a84-9515-359ba0559c6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:19.994703 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157775e6-5aa9-4a84-9515-359ba0559c6e admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:20.008865 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:15:20.052815 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1179/2361] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:15:19 2022] POST /identity/v3/projects => generated 373 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:15:20.054260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e0b7f-c6c4-4f4a-82a1-ab44b6fc1117 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:20.069900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e0b7f-c6c4-4f4a-82a1-ab44b6fc1117 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:20.207959 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282c199-4dc5-4ae7-97ec-3983d165c2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:20.209597 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282c199-4dc5-4ae7-97ec-3983d165c2c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:20.209597 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282c199-4dc5-4ae7-97ec-3983d165c2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:20.210040 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282c199-4dc5-4ae7-97ec-3983d165c2c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:20.211986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282c199-4dc5-4ae7-97ec-3983d165c2c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:20.214586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282c199-4dc5-4ae7-97ec-3983d165c2c3 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:20.257128 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1183/2362] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:15:20 2022] POST /identity/v3/users => generated 287 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:15:20.261423 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157775e6-5aa9-4a84-9515-359ba0559c6e admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:20.273622 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157775e6-5aa9-4a84-9515-359ba0559c6e admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:20.287595 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20098863-2960-4986-a84b-50d3377d6a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:20.288814 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20098863-2960-4986-a84b-50d3377d6a48 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:20.289190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20098863-2960-4986-a84b-50d3377d6a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:20.289634 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20098863-2960-4986-a84b-50d3377d6a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:20.290792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20098863-2960-4986-a84b-50d3377d6a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:20.294744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20098863-2960-4986-a84b-50d3377d6a48 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:20.326326 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1180/2363] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:15:20.329259 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282c199-4dc5-4ae7-97ec-3983d165c2c3 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:20.345973 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282c199-4dc5-4ae7-97ec-3983d165c2c3 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:20.355998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606daec-25f0-4114-97c4-7dc8082a78c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:20.357071 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606daec-25f0-4114-97c4-7dc8082a78c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:20.357558 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606daec-25f0-4114-97c4-7dc8082a78c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:20.358211 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606daec-25f0-4114-97c4-7dc8082a78c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:20.359420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8606daec-25f0-4114-97c4-7dc8082a78c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:20.362231 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8606daec-25f0-4114-97c4-7dc8082a78c0 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:20.387080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1184/2364] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:15:20.388851 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20098863-2960-4986-a84b-50d3377d6a48 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:20.456833 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20098863-2960-4986-a84b-50d3377d6a48 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:20.481257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8bf9d-d66f-4535-b81e-44124bbb2531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:20.481585 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8bf9d-d66f-4535-b81e-44124bbb2531 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:20.482008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8bf9d-d66f-4535-b81e-44124bbb2531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:20.482504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8bf9d-d66f-4535-b81e-44124bbb2531 admin admin] PATH_INFO: `/v3/projects/98b4b754e82445a0b65e88f5dbede11b/users/70c47e7154fd41a7adb4735a519f1484/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:20.536495 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8bf9d-d66f-4535-b81e-44124bbb2531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b4b754e82445a0b65e88f5dbede11b, user_id=70c47e7154fd41a7adb4735a519f1484, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:20.542345 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8bf9d-d66f-4535-b81e-44124bbb2531 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:20.574809 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1181/2365] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:15:20 2022] PUT /identity/v3/projects/98b4b754e82445a0b65e88f5dbede11b/users/70c47e7154fd41a7adb4735a519f1484/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:15:20.583869 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc088d1-e0d3-4c4d-b4fb-f72b7aaa02e6 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:20.584773 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc088d1-e0d3-4c4d-b4fb-f72b7aaa02e6 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:20.585502 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc088d1-e0d3-4c4d-b4fb-f72b7aaa02e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:20.629658 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-2fc088d1-e0d3-4c4d-b4fb-f72b7aaa02e6 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:15:20.631659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-2fc088d1-e0d3-4c4d-b4fb-f72b7aaa02e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:20.774520 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1185/2366] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:15:20.790395 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8bf9d-d66f-4535-b81e-44124bbb2531 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:20.793304 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-12e8bf9d-d66f-4535-b81e-44124bbb2531 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:20.896149 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8bf9d-d66f-4535-b81e-44124bbb2531 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:20.902738 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c0f89-df66-43a1-bc78-4822706cf87b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:20.903786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c0f89-df66-43a1-bc78-4822706cf87b service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:20.904271 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c0f89-df66-43a1-bc78-4822706cf87b service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:20.905632 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c0f89-df66-43a1-bc78-4822706cf87b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:20.923542 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743c0f89-df66-43a1-bc78-4822706cf87b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:20.925345 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743c0f89-df66-43a1-bc78-4822706cf87b service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:20.944111 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1182/2367] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:15:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:15:30.056674 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f47f3-9b36-4c15-8464-08c9e10363b3 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:30.057594 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f47f3-9b36-4c15-8464-08c9e10363b3 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:30.057808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f47f3-9b36-4c15-8464-08c9e10363b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:30.096427 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-ae6f47f3-9b36-4c15-8464-08c9e10363b3 None None] MFA Rules not processed for user `70c47e7154fd41a7adb4735a519f1484`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:15:30.098066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ae6f47f3-9b36-4c15-8464-08c9e10363b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:30.122788 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:15:30.203192 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1186/2368] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:15:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:15:30.204825 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c0f89-df66-43a1-bc78-4822706cf87b service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:30.216097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c0f89-df66-43a1-bc78-4822706cf87b service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:30.223414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95524efd-a5bb-4d64-8bd3-215141dfb605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:30.224234 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95524efd-a5bb-4d64-8bd3-215141dfb605 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:30.224665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95524efd-a5bb-4d64-8bd3-215141dfb605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:30.225105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95524efd-a5bb-4d64-8bd3-215141dfb605 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:30.226190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95524efd-a5bb-4d64-8bd3-215141dfb605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:30.229611 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95524efd-a5bb-4d64-8bd3-215141dfb605 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:30.273811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f47f3-9b36-4c15-8464-08c9e10363b3 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:30.276601 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1183/2369] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:15:30 2022] POST /identity/v3/projects => generated 371 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:15:30.294554 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f47f3-9b36-4c15-8464-08c9e10363b3 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:30.423545 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec239158-f486-458f-9c9e-80570419fb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:30.424535 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec239158-f486-458f-9c9e-80570419fb1a admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:30.424966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec239158-f486-458f-9c9e-80570419fb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:30.425503 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec239158-f486-458f-9c9e-80570419fb1a admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:30.432861 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec239158-f486-458f-9c9e-80570419fb1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:30.432861 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec239158-f486-458f-9c9e-80570419fb1a admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:30.471221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1187/2370] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:15:30 2022] POST /identity/v3/users => generated 292 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:15:30.476974 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95524efd-a5bb-4d64-8bd3-215141dfb605 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:30.494614 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95524efd-a5bb-4d64-8bd3-215141dfb605 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:30.502478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6180d5-4971-42ec-8c72-785998942cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:30.503322 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6180d5-4971-42ec-8c72-785998942cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:30.506664 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6180d5-4971-42ec-8c72-785998942cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:30.506664 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6180d5-4971-42ec-8c72-785998942cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:30.506664 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6180d5-4971-42ec-8c72-785998942cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:30.507508 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6180d5-4971-42ec-8c72-785998942cc5 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:30.525824 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1184/2371] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:15:30.532066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec239158-f486-458f-9c9e-80570419fb1a admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:30.546461 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec239158-f486-458f-9c9e-80570419fb1a admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:30.557823 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb898916-044d-4c2a-9276-1e2096eed9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:30.558638 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb898916-044d-4c2a-9276-1e2096eed9cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:30.559135 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb898916-044d-4c2a-9276-1e2096eed9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:30.559507 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb898916-044d-4c2a-9276-1e2096eed9cc admin admin] PATH_INFO: `/v3/projects/07029b1dc2414781a7590b61f60839ee/users/434966086e7849bca9cce516c259babe/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:30.579027 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb898916-044d-4c2a-9276-1e2096eed9cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=07029b1dc2414781a7590b61f60839ee, user_id=434966086e7849bca9cce516c259babe, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:30.582153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb898916-044d-4c2a-9276-1e2096eed9cc admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:30.603379 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1188/2372] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:15:30 2022] PUT /identity/v3/projects/07029b1dc2414781a7590b61f60839ee/users/434966086e7849bca9cce516c259babe/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:15:30.609253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6180d5-4971-42ec-8c72-785998942cc5 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:30.626913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6180d5-4971-42ec-8c72-785998942cc5 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:30.722748 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ffb14-e171-4a00-9e5d-19a9999aae7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:30.723889 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ffb14-e171-4a00-9e5d-19a9999aae7c admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:30.724351 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ffb14-e171-4a00-9e5d-19a9999aae7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:30.724851 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ffb14-e171-4a00-9e5d-19a9999aae7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:30.725938 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ffb14-e171-4a00-9e5d-19a9999aae7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:30.728633 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ffb14-e171-4a00-9e5d-19a9999aae7c admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:30.748165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1185/2373] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:15:30.755106 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb898916-044d-4c2a-9276-1e2096eed9cc admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:30.778153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb898916-044d-4c2a-9276-1e2096eed9cc admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:30.795087 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f179c8-ec88-46f1-9a8e-09ab7bb312a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:30.797367 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f179c8-ec88-46f1-9a8e-09ab7bb312a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:30.798159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f179c8-ec88-46f1-9a8e-09ab7bb312a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:30.798515 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f179c8-ec88-46f1-9a8e-09ab7bb312a0 admin admin] PATH_INFO: `/v3/projects/07029b1dc2414781a7590b61f60839ee/users/434966086e7849bca9cce516c259babe/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:30.801578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f179c8-ec88-46f1-9a8e-09ab7bb312a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07029b1dc2414781a7590b61f60839ee, user_id=434966086e7849bca9cce516c259babe, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:30.806736 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f179c8-ec88-46f1-9a8e-09ab7bb312a0 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:30.825990 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1189/2374] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:15:30 2022] PUT /identity/v3/projects/07029b1dc2414781a7590b61f60839ee/users/434966086e7849bca9cce516c259babe/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:15:38.877094 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee3e6f-df37-40cd-881f-fee18f3f46b9 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:38.878080 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee3e6f-df37-40cd-881f-fee18f3f46b9 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:38.878080 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee3e6f-df37-40cd-881f-fee18f3f46b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:38.928913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-89ee3e6f-df37-40cd-881f-fee18f3f46b9 None None] MFA Rules not processed for user `434966086e7849bca9cce516c259babe`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:15:38.932428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-89ee3e6f-df37-40cd-881f-fee18f3f46b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:38.964323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:15:38.976673 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f179c8-ec88-46f1-9a8e-09ab7bb312a0 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:38.979563 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-48f179c8-ec88-46f1-9a8e-09ab7bb312a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:39.137688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1186/2375] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:15:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:15:39.161051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee3e6f-df37-40cd-881f-fee18f3f46b9 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:39.163529 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-89ee3e6f-df37-40cd-881f-fee18f3f46b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:39.166105 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f179c8-ec88-46f1-9a8e-09ab7bb312a0 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:39.174222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfaed42-a351-43fb-96b3-1e52516ae325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:39.174901 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaed42-a351-43fb-96b3-1e52516ae325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:39.175184 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaed42-a351-43fb-96b3-1e52516ae325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:39.175598 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaed42-a351-43fb-96b3-1e52516ae325 admin admin] PATH_INFO: `/v3/users/00382f6bad10448ebc1d553c6220f1c7` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:39.180559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfaed42-a351-43fb-96b3-1e52516ae325 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00382f6bad10448ebc1d553c6220f1c7)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:39.185084 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfaed42-a351-43fb-96b3-1e52516ae325 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:39.212858 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:15:39.309463 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-dbfaed42-a351-43fb-96b3-1e52516ae325 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00382f6bad10448ebc1d553c6220f1c7'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:39.342009 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee3e6f-df37-40cd-881f-fee18f3f46b9 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:39.349680 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-dbfaed42-a351-43fb-96b3-1e52516ae325 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00382f6bad10448ebc1d553c6220f1c7'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:39.357206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6251518-67d3-405d-975b-d6f680f4f40c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:39.357206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6251518-67d3-405d-975b-d6f680f4f40c service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:39.357206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6251518-67d3-405d-975b-d6f680f4f40c service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:39.357206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6251518-67d3-405d-975b-d6f680f4f40c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:39.367781 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-dbfaed42-a351-43fb-96b3-1e52516ae325 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00382f6bad10448ebc1d553c6220f1c7'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:39.387471 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-dbfaed42-a351-43fb-96b3-1e52516ae325 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00382f6bad10448ebc1d553c6220f1c7'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:39.390929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-dbfaed42-a351-43fb-96b3-1e52516ae325 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00382f6bad10448ebc1d553c6220f1c7'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:39.416011 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1190/2376] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:15:38 2022] DELETE /identity/v3/users/00382f6bad10448ebc1d553c6220f1c7 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:15:39.422831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4004082-0637-4c91-8630-ee48a505e305 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:39.423656 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4004082-0637-4c91-8630-ee48a505e305 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:39.424759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4004082-0637-4c91-8630-ee48a505e305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:39.473239 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-d4004082-0637-4c91-8630-ee48a505e305 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:15:39.484686 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-d4004082-0637-4c91-8630-ee48a505e305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:39.543983 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6251518-67d3-405d-975b-d6f680f4f40c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:39.545811 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6251518-67d3-405d-975b-d6f680f4f40c service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:39.607730 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1187/2377] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:15:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:15:39.704307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1191/2378] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:15:39.732709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6251518-67d3-405d-975b-d6f680f4f40c service nova] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:39.894894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:15:39.902209 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6251518-67d3-405d-975b-d6f680f4f40c service nova] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:39.986346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25214d-e32e-4178-9a12-f48b0c2efac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:39.987344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25214d-e32e-4178-9a12-f48b0c2efac7 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:39.987926 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25214d-e32e-4178-9a12-f48b0c2efac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:39.988401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25214d-e32e-4178-9a12-f48b0c2efac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:40.002445 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c25214d-e32e-4178-9a12-f48b0c2efac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:40.004159 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c25214d-e32e-4178-9a12-f48b0c2efac7 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:40.018654 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1188/2379] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:15:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:15:41.619288 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4004082-0637-4c91-8630-ee48a505e305 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:41.630248 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-d4004082-0637-4c91-8630-ee48a505e305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:41.659587 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4004082-0637-4c91-8630-ee48a505e305 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:41.673578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168670c4-b5da-493e-9488-d441fc250d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:41.678277 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168670c4-b5da-493e-9488-d441fc250d0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:41.679406 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168670c4-b5da-493e-9488-d441fc250d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:41.680229 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168670c4-b5da-493e-9488-d441fc250d0d admin admin] PATH_INFO: `/v3/projects/1d0bc6ba67fb47ddb3948d08e25a137e` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:41.687598 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168670c4-b5da-493e-9488-d441fc250d0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d0bc6ba67fb47ddb3948d08e25a137e)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:41.691537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168670c4-b5da-493e-9488-d441fc250d0d admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:41.784466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-168670c4-b5da-493e-9488-d441fc250d0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d0bc6ba67fb47ddb3948d08e25a137e', 'request_id': 'req-168670c4-b5da-493e-9488-d441fc250d0d'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:41.791837 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-168670c4-b5da-493e-9488-d441fc250d0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d0bc6ba67fb47ddb3948d08e25a137e', 'request_id': 'req-168670c4-b5da-493e-9488-d441fc250d0d'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:41.805743 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-168670c4-b5da-493e-9488-d441fc250d0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d0bc6ba67fb47ddb3948d08e25a137e', 'request_id': 'req-168670c4-b5da-493e-9488-d441fc250d0d'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:41.829464 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-168670c4-b5da-493e-9488-d441fc250d0d admin admin] The token cache is being invalidate because project 1d0bc6ba67fb47ddb3948d08e25a137e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:15:41.829464 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-168670c4-b5da-493e-9488-d441fc250d0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:41.830262 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-168670c4-b5da-493e-9488-d441fc250d0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:15:41.833756 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1192/2380] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:15:41 2022] DELETE /identity/v3/projects/1d0bc6ba67fb47ddb3948d08e25a137e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:15:41.837692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d39e9-3d96-4b89-9a6a-b36a1d5f682c None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:41.838284 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d39e9-3d96-4b89-9a6a-b36a1d5f682c None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:41.838400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d39e9-3d96-4b89-9a6a-b36a1d5f682c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:41.877101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-689d39e9-3d96-4b89-9a6a-b36a1d5f682c None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:15:41.879611 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-689d39e9-3d96-4b89-9a6a-b36a1d5f682c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:42.009687 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1189/2381] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:15:42.013418 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188315c1-1749-4714-bc60-0b0834a852a4 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:42.014030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188315c1-1749-4714-bc60-0b0834a852a4 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:42.015036 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188315c1-1749-4714-bc60-0b0834a852a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:42.046611 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-188315c1-1749-4714-bc60-0b0834a852a4 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:15:42.047885 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-188315c1-1749-4714-bc60-0b0834a852a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:42.062934 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1193/2382] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:15:42.068543 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d39e9-3d96-4b89-9a6a-b36a1d5f682c None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:42.080013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d39e9-3d96-4b89-9a6a-b36a1d5f682c None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:42.085620 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3142223-ab06-46e2-bb5e-dc331bde52cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:42.086513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3142223-ab06-46e2-bb5e-dc331bde52cd admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:42.086941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3142223-ab06-46e2-bb5e-dc331bde52cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:42.088135 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3142223-ab06-46e2-bb5e-dc331bde52cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:42.089013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3142223-ab06-46e2-bb5e-dc331bde52cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:15:42.089321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3142223-ab06-46e2-bb5e-dc331bde52cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:42.091100 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3142223-ab06-46e2-bb5e-dc331bde52cd admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:42.126694 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1190/2383] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:15:42.136062 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188315c1-1749-4714-bc60-0b0834a852a4 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:42.169504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188315c1-1749-4714-bc60-0b0834a852a4 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:42.179829 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf8ef2-6c15-48a9-85f6-7d96a9957f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:42.181934 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf8ef2-6c15-48a9-85f6-7d96a9957f6c admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:42.182728 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf8ef2-6c15-48a9-85f6-7d96a9957f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:42.183494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf8ef2-6c15-48a9-85f6-7d96a9957f6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:42.185959 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf8ef2-6c15-48a9-85f6-7d96a9957f6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:42.192077 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf8ef2-6c15-48a9-85f6-7d96a9957f6c admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:42.203305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:15:42.287679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1194/2384] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:15:42 2022] POST /identity/v3/projects => generated 363 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:15:42.293406 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3142223-ab06-46e2-bb5e-dc331bde52cd admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:42.323640 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3142223-ab06-46e2-bb5e-dc331bde52cd admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:42.493846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44685fea-bbaf-46c9-930f-e349605c6b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:42.495171 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44685fea-bbaf-46c9-930f-e349605c6b4d admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:42.495628 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44685fea-bbaf-46c9-930f-e349605c6b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:42.496131 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44685fea-bbaf-46c9-930f-e349605c6b4d admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:42.497970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44685fea-bbaf-46c9-930f-e349605c6b4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:42.503727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44685fea-bbaf-46c9-930f-e349605c6b4d admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:42.545621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1191/2385] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:15:42 2022] POST /identity/v3/users => generated 282 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:15:42.550975 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf8ef2-6c15-48a9-85f6-7d96a9957f6c admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:42.564717 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf8ef2-6c15-48a9-85f6-7d96a9957f6c admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:42.573632 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90024fac-eebe-491b-a703-a91ed89107df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:42.574376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90024fac-eebe-491b-a703-a91ed89107df admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:42.574707 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90024fac-eebe-491b-a703-a91ed89107df admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:42.575356 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90024fac-eebe-491b-a703-a91ed89107df admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:42.575922 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90024fac-eebe-491b-a703-a91ed89107df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:42.578819 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90024fac-eebe-491b-a703-a91ed89107df admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:42.595894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1195/2386] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:15:42.603935 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44685fea-bbaf-46c9-930f-e349605c6b4d admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:42.622315 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44685fea-bbaf-46c9-930f-e349605c6b4d admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:42.628282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ba4cb-0c15-4d1b-9a89-1ed7fc71854f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:42.629306 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ba4cb-0c15-4d1b-9a89-1ed7fc71854f admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:42.629637 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ba4cb-0c15-4d1b-9a89-1ed7fc71854f admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:42.630083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ba4cb-0c15-4d1b-9a89-1ed7fc71854f admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:42.630711 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ba4cb-0c15-4d1b-9a89-1ed7fc71854f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:42.632704 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ba4cb-0c15-4d1b-9a89-1ed7fc71854f admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:42.661856 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1192/2387] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:15:42.664577 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90024fac-eebe-491b-a703-a91ed89107df admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:42.677011 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90024fac-eebe-491b-a703-a91ed89107df admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:42.682969 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c114c9-0173-4ae8-94dd-8608d04d098d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:42.683996 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c114c9-0173-4ae8-94dd-8608d04d098d admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:42.684653 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c114c9-0173-4ae8-94dd-8608d04d098d admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:42.684653 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c114c9-0173-4ae8-94dd-8608d04d098d admin admin] PATH_INFO: `/v3/projects/1ed75b7cb6004f59bd51b99bea4f722a/users/02ae3765716f41c78cadd4b1267cd25c/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:42.710111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c114c9-0173-4ae8-94dd-8608d04d098d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ed75b7cb6004f59bd51b99bea4f722a, user_id=02ae3765716f41c78cadd4b1267cd25c, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:42.714221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c114c9-0173-4ae8-94dd-8608d04d098d admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:42.755425 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1196/2388] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:15:42 2022] PUT /identity/v3/projects/1ed75b7cb6004f59bd51b99bea4f722a/users/02ae3765716f41c78cadd4b1267cd25c/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:15:42.775291 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e65f49-58e1-4d8c-9b7e-5cd083246a36 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:42.775645 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e65f49-58e1-4d8c-9b7e-5cd083246a36 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:42.775911 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e65f49-58e1-4d8c-9b7e-5cd083246a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:42.800946 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:15:42.835414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-a2e65f49-58e1-4d8c-9b7e-5cd083246a36 None None] MFA Rules not processed for user `02ae3765716f41c78cadd4b1267cd25c`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:15:42.838305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-a2e65f49-58e1-4d8c-9b7e-5cd083246a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:42.951465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1193/2389] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:15:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:15:42.973678 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c114c9-0173-4ae8-94dd-8608d04d098d admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:42.975929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a5c114c9-0173-4ae8-94dd-8608d04d098d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:43.056681 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87781da-68e0-4eb2-b73b-801d737f8123 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:43.056961 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87781da-68e0-4eb2-b73b-801d737f8123 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:43.057328 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87781da-68e0-4eb2-b73b-801d737f8123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:43.091737 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-f87781da-68e0-4eb2-b73b-801d737f8123 None None] MFA Rules not processed for user `70c47e7154fd41a7adb4735a519f1484`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:15:43.094332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-f87781da-68e0-4eb2-b73b-801d737f8123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:43.108829 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c114c9-0173-4ae8-94dd-8608d04d098d admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:43.114380 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f3d64-a519-4ccc-a4e7-3f6ae1a325f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:43.115178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f3d64-a519-4ccc-a4e7-3f6ae1a325f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:43.115611 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f3d64-a519-4ccc-a4e7-3f6ae1a325f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:43.116176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f3d64-a519-4ccc-a4e7-3f6ae1a325f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:43.130052 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f3d64-a519-4ccc-a4e7-3f6ae1a325f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:43.131544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f3d64-a519-4ccc-a4e7-3f6ae1a325f5 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:43.143786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1197/2390] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:15:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:15:43.203349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1194/2391] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:15:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:15:43.540834 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f3d64-a519-4ccc-a4e7-3f6ae1a325f5 service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:43.543068 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-000f3d64-a519-4ccc-a4e7-3f6ae1a325f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:43.830862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f3d64-a519-4ccc-a4e7-3f6ae1a325f5 service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:43.858687 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f7f94-2108-4a26-889f-84edbb093f9f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab2e318b2854c46bc7ed2151a44a2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:43.859775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f7f94-2108-4a26-889f-84edbb093f9f service placement] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:43.860240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f7f94-2108-4a26-889f-84edbb093f9f service placement] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:43.861445 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f7f94-2108-4a26-889f-84edbb093f9f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:43.865825 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a10f7f94-2108-4a26-889f-84edbb093f9f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:43.924087 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f7f94-2108-4a26-889f-84edbb093f9f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:43.932459 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f7f94-2108-4a26-889f-84edbb093f9f service placement] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:44.012181 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1198/2392] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:15:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:15:51.371121 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705c1df-7239-4c07-91de-06198937da41 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:51.371121 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705c1df-7239-4c07-91de-06198937da41 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:51.372116 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705c1df-7239-4c07-91de-06198937da41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:51.398810 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-7705c1df-7239-4c07-91de-06198937da41 None None] MFA Rules not processed for user `e0c76478b6694782b896cdf9cda32d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:15:51.400013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-7705c1df-7239-4c07-91de-06198937da41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:51.477582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1195/2393] 10.222.0.73 () {58 vars in 1009 bytes} [Mon Aug 29 20:15:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:15:51.545175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f7f94-2108-4a26-889f-84edbb093f9f service placement] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:15:51.548138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a10f7f94-2108-4a26-889f-84edbb093f9f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:15:51.634504 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:15:51.678196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f7f94-2108-4a26-889f-84edbb093f9f service placement] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:15:51.688575 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087947e-ed73-48a3-9222-8444cad82069 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92e30da8f394f3abc19b9e86ef126e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:15:51.689463 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087947e-ed73-48a3-9222-8444cad82069 service swift] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:15:51.690006 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087947e-ed73-48a3-9222-8444cad82069 service swift] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:15:51.690444 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087947e-ed73-48a3-9222-8444cad82069 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:15:51.707955 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b087947e-ed73-48a3-9222-8444cad82069 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:15:51.712743 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b087947e-ed73-48a3-9222-8444cad82069 service swift] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:15:51.730298 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1199/2394] 10.222.0.73 () {60 vars in 1384 bytes} [Mon Aug 29 20:15:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:16:01.468979 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=79 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:16:01.724431 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=13 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:16:29.637648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c6860-b625-455c-a434-97aa7773fcd5 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:29.638318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c6860-b625-455c-a434-97aa7773fcd5 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:29.638587 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c6860-b625-455c-a434-97aa7773fcd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:29.668281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-909c6860-b625-455c-a434-97aa7773fcd5 None None] MFA Rules not processed for user `e0c76478b6694782b896cdf9cda32d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:16:29.670392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-909c6860-b625-455c-a434-97aa7773fcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:16:29.682491 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1196/2395] 10.222.0.73 () {58 vars in 967 bytes} [Mon Aug 29 20:16:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:16:29.704044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087947e-ed73-48a3-9222-8444cad82069 service swift] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:16:29.722086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087947e-ed73-48a3-9222-8444cad82069 service swift] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:16:29.730633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e9860-fab2-4d58-9957-851edf45ba83 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92e30da8f394f3abc19b9e86ef126e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:16:29.731610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e9860-fab2-4d58-9957-851edf45ba83 service swift] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:29.732088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e9860-fab2-4d58-9957-851edf45ba83 service swift] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:29.732492 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e9860-fab2-4d58-9957-851edf45ba83 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:29.745209 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e9860-fab2-4d58-9957-851edf45ba83 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:16:29.747002 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e9860-fab2-4d58-9957-851edf45ba83 service swift] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:16:29.758262 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1200/2396] 10.222.0.73 () {60 vars in 1384 bytes} [Mon Aug 29 20:16:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:16:32.306891 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfa261-1ec1-4893-b715-b6647bc706ec None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:32.307704 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfa261-1ec1-4893-b715-b6647bc706ec None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:32.307944 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfa261-1ec1-4893-b715-b6647bc706ec None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:32.310389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1197/2397] 10.222.0.73 () {54 vars in 921 bytes} [Mon Aug 29 20:16:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:16:32.348730 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7060d-537c-4a5e-a700-4658e6626359 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:32.349611 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7060d-537c-4a5e-a700-4658e6626359 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:32.349611 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7060d-537c-4a5e-a700-4658e6626359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:32.375303 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-aae7060d-537c-4a5e-a700-4658e6626359 None None] MFA Rules not processed for user `1ab2e318b2854c46bc7ed2151a44a2e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:16:32.379365 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-aae7060d-537c-4a5e-a700-4658e6626359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:16:32.394485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1201/2398] 10.222.0.73 () {58 vars in 1018 bytes} [Mon Aug 29 20:16:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:16:32.413279 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfa261-1ec1-4893-b715-b6647bc706ec None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:16:32.416479 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7060d-537c-4a5e-a700-4658e6626359 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:16:32.416757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9ecfa261-1ec1-4893-b715-b6647bc706ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:16:32.423558 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-aae7060d-537c-4a5e-a700-4658e6626359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:16:32.452820 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfa261-1ec1-4893-b715-b6647bc706ec None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:16:32.462391 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3864d-8779-46cf-952c-44ba1d0714a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab2e318b2854c46bc7ed2151a44a2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:16:32.463942 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3864d-8779-46cf-952c-44ba1d0714a9 service placement] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:32.464678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3864d-8779-46cf-952c-44ba1d0714a9 service placement] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:32.464950 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3864d-8779-46cf-952c-44ba1d0714a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:32.482621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3864d-8779-46cf-952c-44ba1d0714a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:16:32.486833 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3864d-8779-46cf-952c-44ba1d0714a9 service placement] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:16:32.502251 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1198/2399] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:16:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:16:32.614194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7060d-537c-4a5e-a700-4658e6626359 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:16:32.620016 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11bd69-639c-42c1-aa9e-42826927b0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:16:32.621059 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11bd69-639c-42c1-aa9e-42826927b0e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:32.621455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11bd69-639c-42c1-aa9e-42826927b0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:32.622157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11bd69-639c-42c1-aa9e-42826927b0e3 admin admin] PATH_INFO: `/v3/users/be17fa7c5ebe49efa4bcae7c721a5e01` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:32.624305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11bd69-639c-42c1-aa9e-42826927b0e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be17fa7c5ebe49efa4bcae7c721a5e01)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:16:32.627662 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11bd69-639c-42c1-aa9e-42826927b0e3 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:16:32.728348 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-bb11bd69-639c-42c1-aa9e-42826927b0e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be17fa7c5ebe49efa4bcae7c721a5e01'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:16:32.759009 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-bb11bd69-639c-42c1-aa9e-42826927b0e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be17fa7c5ebe49efa4bcae7c721a5e01'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:16:32.779723 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-bb11bd69-639c-42c1-aa9e-42826927b0e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be17fa7c5ebe49efa4bcae7c721a5e01'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:16:32.798531 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-bb11bd69-639c-42c1-aa9e-42826927b0e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be17fa7c5ebe49efa4bcae7c721a5e01'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:16:32.799764 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-bb11bd69-639c-42c1-aa9e-42826927b0e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be17fa7c5ebe49efa4bcae7c721a5e01'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:16:32.811011 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1202/2400] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:16:32 2022] DELETE /identity/v3/users/be17fa7c5ebe49efa4bcae7c721a5e01 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:16:33.149626 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3864d-8779-46cf-952c-44ba1d0714a9 service placement] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:16:33.172801 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3864d-8779-46cf-952c-44ba1d0714a9 service placement] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:16:33.273149 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8feb8-7842-469c-95bb-6060e453d3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:16:33.274290 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8feb8-7842-469c-95bb-6060e453d3c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:33.274791 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8feb8-7842-469c-95bb-6060e453d3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:33.275240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8feb8-7842-469c-95bb-6060e453d3c4 admin admin] PATH_INFO: `/v3/projects/95b900960ce84ebba14e9157e030287e` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:33.277541 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b8feb8-7842-469c-95bb-6060e453d3c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95b900960ce84ebba14e9157e030287e)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:16:33.280142 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b8feb8-7842-469c-95bb-6060e453d3c4 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:16:33.367029 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c6b8feb8-7842-469c-95bb-6060e453d3c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95b900960ce84ebba14e9157e030287e', 'request_id': 'req-c6b8feb8-7842-469c-95bb-6060e453d3c4'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:16:33.379187 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c6b8feb8-7842-469c-95bb-6060e453d3c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95b900960ce84ebba14e9157e030287e', 'request_id': 'req-c6b8feb8-7842-469c-95bb-6060e453d3c4'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:16:33.387658 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c6b8feb8-7842-469c-95bb-6060e453d3c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95b900960ce84ebba14e9157e030287e', 'request_id': 'req-c6b8feb8-7842-469c-95bb-6060e453d3c4'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:16:33.409644 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c6b8feb8-7842-469c-95bb-6060e453d3c4 admin admin] The token cache is being invalidate because project 95b900960ce84ebba14e9157e030287e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:16:33.410078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c6b8feb8-7842-469c-95bb-6060e453d3c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:16:33.410926 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c6b8feb8-7842-469c-95bb-6060e453d3c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:16:33.412628 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1199/2401] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:16:33 2022] DELETE /identity/v3/projects/95b900960ce84ebba14e9157e030287e => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:16:33.418854 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5677ab-06eb-4600-9d5d-2077b3932b39 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:33.419483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5677ab-06eb-4600-9d5d-2077b3932b39 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:33.419622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5677ab-06eb-4600-9d5d-2077b3932b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:33.449664 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-7d5677ab-06eb-4600-9d5d-2077b3932b39 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:16:33.451828 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-7d5677ab-06eb-4600-9d5d-2077b3932b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:16:33.527942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:16:33.543146 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1203/2402] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:16:33.552447 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8eb94-610d-4732-b33b-8225cf1ecdd3 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:33.552865 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8eb94-610d-4732-b33b-8225cf1ecdd3 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:33.553241 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8eb94-610d-4732-b33b-8225cf1ecdd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:33.583017 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-a6e8eb94-610d-4732-b33b-8225cf1ecdd3 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:16:33.584035 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-a6e8eb94-610d-4732-b33b-8225cf1ecdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:16:33.595335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1200/2403] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:16:33.600393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5677ab-06eb-4600-9d5d-2077b3932b39 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:16:33.622273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5677ab-06eb-4600-9d5d-2077b3932b39 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:16:33.631816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66c587-c06e-4dd9-a3f0-52459ff51666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:16:33.633237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66c587-c06e-4dd9-a3f0-52459ff51666 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:33.633592 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66c587-c06e-4dd9-a3f0-52459ff51666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:33.634215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66c587-c06e-4dd9-a3f0-52459ff51666 admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:33.635500 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66c587-c06e-4dd9-a3f0-52459ff51666 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:16:33.635991 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66c587-c06e-4dd9-a3f0-52459ff51666 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:16:33.638759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66c587-c06e-4dd9-a3f0-52459ff51666 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:16:33.659837 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1204/2404] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:16:33.667290 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8eb94-610d-4732-b33b-8225cf1ecdd3 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:16:33.684883 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8eb94-610d-4732-b33b-8225cf1ecdd3 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:16:33.692363 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13caede-6a66-4f9d-bd32-09699c89cfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:16:33.693114 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13caede-6a66-4f9d-bd32-09699c89cfa7 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:33.693395 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13caede-6a66-4f9d-bd32-09699c89cfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:33.693747 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13caede-6a66-4f9d-bd32-09699c89cfa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:33.695017 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13caede-6a66-4f9d-bd32-09699c89cfa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:16:33.698318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13caede-6a66-4f9d-bd32-09699c89cfa7 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:16:33.750976 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1201/2405] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:16:33 2022] POST /identity/v3/projects => generated 369 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:16:33.761764 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66c587-c06e-4dd9-a3f0-52459ff51666 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:16:33.770703 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66c587-c06e-4dd9-a3f0-52459ff51666 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:16:33.900439 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a877cf-fbfd-488d-ba14-11fc79784b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:16:33.901584 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a877cf-fbfd-488d-ba14-11fc79784b8a admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:33.902059 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a877cf-fbfd-488d-ba14-11fc79784b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:33.902540 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a877cf-fbfd-488d-ba14-11fc79784b8a admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:33.903888 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a877cf-fbfd-488d-ba14-11fc79784b8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:16:33.909083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a877cf-fbfd-488d-ba14-11fc79784b8a admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:16:33.949771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1205/2406] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:16:33 2022] POST /identity/v3/users => generated 285 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:16:33.952337 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13caede-6a66-4f9d-bd32-09699c89cfa7 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:16:33.965878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13caede-6a66-4f9d-bd32-09699c89cfa7 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:16:33.974340 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265b781-982b-4483-b99f-73d956bcce5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:16:33.975343 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265b781-982b-4483-b99f-73d956bcce5f admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:33.975686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265b781-982b-4483-b99f-73d956bcce5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:33.976000 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265b781-982b-4483-b99f-73d956bcce5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:33.976882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265b781-982b-4483-b99f-73d956bcce5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:16:33.978801 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265b781-982b-4483-b99f-73d956bcce5f admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:16:33.999510 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1202/2407] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:16:34.002227 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a877cf-fbfd-488d-ba14-11fc79784b8a admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:16:34.023277 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a877cf-fbfd-488d-ba14-11fc79784b8a admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:16:34.034661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3588b9e-a15f-4a33-90d1-db54f794d394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:16:34.035335 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3588b9e-a15f-4a33-90d1-db54f794d394 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:34.035781 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3588b9e-a15f-4a33-90d1-db54f794d394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:34.036380 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3588b9e-a15f-4a33-90d1-db54f794d394 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:34.037528 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3588b9e-a15f-4a33-90d1-db54f794d394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:16:34.043978 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3588b9e-a15f-4a33-90d1-db54f794d394 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:16:34.061750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1206/2408] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:16:34.072142 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265b781-982b-4483-b99f-73d956bcce5f admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:16:34.093236 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265b781-982b-4483-b99f-73d956bcce5f admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:16:34.119204 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0527fd-cd5e-45ee-a628-8d2b596648ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:16:34.120946 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0527fd-cd5e-45ee-a628-8d2b596648ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:34.120946 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0527fd-cd5e-45ee-a628-8d2b596648ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:34.120946 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0527fd-cd5e-45ee-a628-8d2b596648ee admin admin] PATH_INFO: `/v3/projects/58302c93bbd949569c1bd3e758a023b8/users/542eb802a8584c0a9ab1edd3e8fa41ee/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:34.153437 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0527fd-cd5e-45ee-a628-8d2b596648ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=58302c93bbd949569c1bd3e758a023b8, user_id=542eb802a8584c0a9ab1edd3e8fa41ee, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:16:34.158881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0527fd-cd5e-45ee-a628-8d2b596648ee admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:16:34.183452 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1203/2409] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:16:34 2022] PUT /identity/v3/projects/58302c93bbd949569c1bd3e758a023b8/users/542eb802a8584c0a9ab1edd3e8fa41ee/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:16:34.196594 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a07d2-2c13-4181-9d92-5f22fa50fa8a None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:34.197014 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a07d2-2c13-4181-9d92-5f22fa50fa8a None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:34.197511 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a07d2-2c13-4181-9d92-5f22fa50fa8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:34.246373 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-e20a07d2-2c13-4181-9d92-5f22fa50fa8a None None] MFA Rules not processed for user `542eb802a8584c0a9ab1edd3e8fa41ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:16:34.247393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-e20a07d2-2c13-4181-9d92-5f22fa50fa8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:16:34.348669 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1207/2410] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:16:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:16:34.370504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0527fd-cd5e-45ee-a628-8d2b596648ee admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:16:34.370504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-7c0527fd-cd5e-45ee-a628-8d2b596648ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:16:34.379959 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:16:34.455423 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0527fd-cd5e-45ee-a628-8d2b596648ee admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:16:34.460423 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7964bbe0-d9be-491a-8c53-4f2ec5a59bb4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b64b4b172467692d02851b2955b81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:16:34.461390 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964bbe0-d9be-491a-8c53-4f2ec5a59bb4 service glance] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:34.461797 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964bbe0-d9be-491a-8c53-4f2ec5a59bb4 service glance] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:34.462223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964bbe0-d9be-491a-8c53-4f2ec5a59bb4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:34.474341 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7964bbe0-d9be-491a-8c53-4f2ec5a59bb4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:16:34.475998 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7964bbe0-d9be-491a-8c53-4f2ec5a59bb4 service glance] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:16:34.490524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1204/2411] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:16:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:16:44.345393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=72,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:16:44.482784 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=23 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:16:49.375302 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a07d2-2c13-4181-9d92-5f22fa50fa8a None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:16:49.380616 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-e20a07d2-2c13-4181-9d92-5f22fa50fa8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:16:49.479001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a07d2-2c13-4181-9d92-5f22fa50fa8a None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:16:49.487195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3373955-07f5-4c4a-b0a3-5a1914c8b13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:16:49.487883 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3373955-07f5-4c4a-b0a3-5a1914c8b13e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:49.488185 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3373955-07f5-4c4a-b0a3-5a1914c8b13e admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:49.488458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3373955-07f5-4c4a-b0a3-5a1914c8b13e admin admin] PATH_INFO: `/v3/users/02ae3765716f41c78cadd4b1267cd25c` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:49.489818 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3373955-07f5-4c4a-b0a3-5a1914c8b13e admin admin] RBAC: Authorizing `identity:delete_user(user_id=02ae3765716f41c78cadd4b1267cd25c)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:16:49.491823 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3373955-07f5-4c4a-b0a3-5a1914c8b13e admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:16:49.572134 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e3373955-07f5-4c4a-b0a3-5a1914c8b13e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02ae3765716f41c78cadd4b1267cd25c'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:16:49.593166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e3373955-07f5-4c4a-b0a3-5a1914c8b13e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02ae3765716f41c78cadd4b1267cd25c'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:16:49.603186 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e3373955-07f5-4c4a-b0a3-5a1914c8b13e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02ae3765716f41c78cadd4b1267cd25c'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:16:49.619027 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e3373955-07f5-4c4a-b0a3-5a1914c8b13e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02ae3765716f41c78cadd4b1267cd25c'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:16:49.620726 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e3373955-07f5-4c4a-b0a3-5a1914c8b13e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02ae3765716f41c78cadd4b1267cd25c'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:16:49.633246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1208/2412] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:16:49 2022] DELETE /identity/v3/users/02ae3765716f41c78cadd4b1267cd25c => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:16:49.637463 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3faf98-8ed1-48ef-a28a-6309441e665d None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:49.638454 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3faf98-8ed1-48ef-a28a-6309441e665d None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:49.638776 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3faf98-8ed1-48ef-a28a-6309441e665d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:49.667010 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-cb3faf98-8ed1-48ef-a28a-6309441e665d None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:16:49.669010 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-cb3faf98-8ed1-48ef-a28a-6309441e665d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:16:49.780459 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1205/2413] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:16:49.797062 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3373955-07f5-4c4a-b0a3-5a1914c8b13e admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:16:49.799265 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-e3373955-07f5-4c4a-b0a3-5a1914c8b13e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:16:49.891439 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3373955-07f5-4c4a-b0a3-5a1914c8b13e admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:16:49.896289 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a0b84-25c7-4314-89cb-b116b5d2efac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:16:49.897582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a0b84-25c7-4314-89cb-b116b5d2efac service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:49.898130 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a0b84-25c7-4314-89cb-b116b5d2efac service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:49.898603 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a0b84-25c7-4314-89cb-b116b5d2efac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:49.915732 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a0b84-25c7-4314-89cb-b116b5d2efac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:16:49.918162 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a0b84-25c7-4314-89cb-b116b5d2efac service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:16:49.935105 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1209/2414] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:16:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:16:50.504851 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3faf98-8ed1-48ef-a28a-6309441e665d None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:16:50.507495 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-cb3faf98-8ed1-48ef-a28a-6309441e665d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:16:50.540621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3faf98-8ed1-48ef-a28a-6309441e665d None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:16:50.551193 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9a9c1-d3c0-4f23-9155-572d05ccd93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:16:50.552653 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9a9c1-d3c0-4f23-9155-572d05ccd93b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:50.553282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9a9c1-d3c0-4f23-9155-572d05ccd93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:50.553921 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9a9c1-d3c0-4f23-9155-572d05ccd93b admin admin] PATH_INFO: `/v3/projects/1ed75b7cb6004f59bd51b99bea4f722a` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:50.556528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9a9c1-d3c0-4f23-9155-572d05ccd93b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ed75b7cb6004f59bd51b99bea4f722a)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:16:50.560563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9a9c1-d3c0-4f23-9155-572d05ccd93b admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:16:50.680847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-cbb9a9c1-d3c0-4f23-9155-572d05ccd93b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ed75b7cb6004f59bd51b99bea4f722a', 'request_id': 'req-cbb9a9c1-d3c0-4f23-9155-572d05ccd93b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:16:50.693796 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-cbb9a9c1-d3c0-4f23-9155-572d05ccd93b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ed75b7cb6004f59bd51b99bea4f722a', 'request_id': 'req-cbb9a9c1-d3c0-4f23-9155-572d05ccd93b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:16:50.738011 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-cbb9a9c1-d3c0-4f23-9155-572d05ccd93b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ed75b7cb6004f59bd51b99bea4f722a', 'request_id': 'req-cbb9a9c1-d3c0-4f23-9155-572d05ccd93b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:16:50.756368 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-cbb9a9c1-d3c0-4f23-9155-572d05ccd93b admin admin] The token cache is being invalidate because project 1ed75b7cb6004f59bd51b99bea4f722a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:16:50.756712 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-cbb9a9c1-d3c0-4f23-9155-572d05ccd93b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:16:50.758155 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-cbb9a9c1-d3c0-4f23-9155-572d05ccd93b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:16:50.761828 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1206/2415] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:16:50 2022] DELETE /identity/v3/projects/1ed75b7cb6004f59bd51b99bea4f722a => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:16:50.770184 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ed388-dbc1-4787-853c-87674adabc37 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:50.770746 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ed388-dbc1-4787-853c-87674adabc37 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:50.771117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ed388-dbc1-4787-853c-87674adabc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:50.815710 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-0e1ed388-dbc1-4787-853c-87674adabc37 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:16:50.817987 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-0e1ed388-dbc1-4787-853c-87674adabc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:16:50.844662 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:16:50.927144 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1210/2416] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:16:50.940701 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77007724-96b3-46cd-b3dc-767649ba8cd3 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:50.941134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77007724-96b3-46cd-b3dc-767649ba8cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:50.941685 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77007724-96b3-46cd-b3dc-767649ba8cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:50.975276 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-77007724-96b3-46cd-b3dc-767649ba8cd3 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:16:50.977606 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-77007724-96b3-46cd-b3dc-767649ba8cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:16:50.993417 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1207/2417] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:16:51.000705 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ed388-dbc1-4787-853c-87674adabc37 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:16:51.027837 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ed388-dbc1-4787-853c-87674adabc37 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:16:51.036099 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603624e-4d54-4849-8316-22305e06c3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:16:51.037195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603624e-4d54-4849-8316-22305e06c3e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:51.037851 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603624e-4d54-4849-8316-22305e06c3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:51.038372 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603624e-4d54-4849-8316-22305e06c3e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:51.039618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603624e-4d54-4849-8316-22305e06c3e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:16:51.039963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603624e-4d54-4849-8316-22305e06c3e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:16:51.043702 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603624e-4d54-4849-8316-22305e06c3e9 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:16:51.069286 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1211/2418] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:16:51.075851 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77007724-96b3-46cd-b3dc-767649ba8cd3 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:16:51.095241 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77007724-96b3-46cd-b3dc-767649ba8cd3 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:16:51.103036 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d740ca-a6ef-4a1d-a8be-1951db0ac5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:16:51.103730 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d740ca-a6ef-4a1d-a8be-1951db0ac5b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:51.104180 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d740ca-a6ef-4a1d-a8be-1951db0ac5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:51.109760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d740ca-a6ef-4a1d-a8be-1951db0ac5b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:51.109760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d740ca-a6ef-4a1d-a8be-1951db0ac5b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:16:51.109760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d740ca-a6ef-4a1d-a8be-1951db0ac5b5 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:16:51.163847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1208/2419] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:16:51 2022] POST /identity/v3/projects => generated 373 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:16:51.170171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603624e-4d54-4849-8316-22305e06c3e9 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:16:51.198559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603624e-4d54-4849-8316-22305e06c3e9 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:16:51.322668 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ca054-1522-4a42-b7a2-6040e0ad381e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:16:51.324000 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ca054-1522-4a42-b7a2-6040e0ad381e admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:51.324384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ca054-1522-4a42-b7a2-6040e0ad381e admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:51.324852 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ca054-1522-4a42-b7a2-6040e0ad381e admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:51.326025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ca054-1522-4a42-b7a2-6040e0ad381e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:16:51.329068 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ca054-1522-4a42-b7a2-6040e0ad381e admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:16:51.366297 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1212/2420] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:16:51 2022] POST /identity/v3/users => generated 287 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:16:51.371041 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d740ca-a6ef-4a1d-a8be-1951db0ac5b5 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:16:51.388778 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d740ca-a6ef-4a1d-a8be-1951db0ac5b5 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:16:51.398249 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75347bfc-8f0e-4091-bf60-e75b6cf11406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:16:51.399841 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75347bfc-8f0e-4091-bf60-e75b6cf11406 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:51.400340 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75347bfc-8f0e-4091-bf60-e75b6cf11406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:51.401027 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75347bfc-8f0e-4091-bf60-e75b6cf11406 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:51.402196 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75347bfc-8f0e-4091-bf60-e75b6cf11406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:16:51.403956 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75347bfc-8f0e-4091-bf60-e75b6cf11406 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:16:51.427956 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1209/2421] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:16:51.434764 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ca054-1522-4a42-b7a2-6040e0ad381e admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:16:51.449202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ca054-1522-4a42-b7a2-6040e0ad381e admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:16:51.455393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e92d84-89a0-42da-a781-d4c7b4fefdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:16:51.456252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e92d84-89a0-42da-a781-d4c7b4fefdc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:51.456526 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e92d84-89a0-42da-a781-d4c7b4fefdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:51.456803 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e92d84-89a0-42da-a781-d4c7b4fefdc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:51.457537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e92d84-89a0-42da-a781-d4c7b4fefdc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:16:51.459325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e92d84-89a0-42da-a781-d4c7b4fefdc9 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:16:51.478236 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1213/2422] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:16:51.483051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75347bfc-8f0e-4091-bf60-e75b6cf11406 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:16:51.496221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75347bfc-8f0e-4091-bf60-e75b6cf11406 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:16:51.509694 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8c871-2245-4109-be7c-d5413dab78ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:16:51.509694 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8c871-2245-4109-be7c-d5413dab78ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:51.509694 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8c871-2245-4109-be7c-d5413dab78ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:51.509694 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8c871-2245-4109-be7c-d5413dab78ef admin admin] PATH_INFO: `/v3/projects/1b6e748b13ea4fc5ad88eaee50ffa437/users/b41427ee7f3a4b478f5587b41a3b9e5e/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:51.540082 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8c871-2245-4109-be7c-d5413dab78ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b6e748b13ea4fc5ad88eaee50ffa437, user_id=b41427ee7f3a4b478f5587b41a3b9e5e, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:16:51.543686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8c871-2245-4109-be7c-d5413dab78ef admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:16:51.563109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1210/2423] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:16:51 2022] PUT /identity/v3/projects/1b6e748b13ea4fc5ad88eaee50ffa437/users/b41427ee7f3a4b478f5587b41a3b9e5e/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:16:51.651891 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b8d73-abdb-4c66-ad6e-0b9674462a3f None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:51.652408 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b8d73-abdb-4c66-ad6e-0b9674462a3f None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:51.652696 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b8d73-abdb-4c66-ad6e-0b9674462a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:51.701272 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-e48b8d73-abdb-4c66-ad6e-0b9674462a3f None None] MFA Rules not processed for user `b41427ee7f3a4b478f5587b41a3b9e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:16:51.704191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-e48b8d73-abdb-4c66-ad6e-0b9674462a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:16:51.839936 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1214/2424] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:16:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:16:51.857073 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8c871-2245-4109-be7c-d5413dab78ef admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:16:51.861381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-acf8c871-2245-4109-be7c-d5413dab78ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:16:51.907773 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:16:51.987465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8c871-2245-4109-be7c-d5413dab78ef admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:16:52.002619 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f804a-398d-4137-a1f1-cc7134d343fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:16:52.003604 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f804a-398d-4137-a1f1-cc7134d343fd service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:52.004487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f804a-398d-4137-a1f1-cc7134d343fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:52.005802 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f804a-398d-4137-a1f1-cc7134d343fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:52.017479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f804a-398d-4137-a1f1-cc7134d343fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:16:52.019715 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f804a-398d-4137-a1f1-cc7134d343fd service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:16:52.033855 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1211/2425] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:16:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:16:53.059848 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fd296-11f5-4180-b82c-d234e9a173ad None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:53.060595 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fd296-11f5-4180-b82c-d234e9a173ad None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:53.060787 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fd296-11f5-4180-b82c-d234e9a173ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:53.096250 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-eb8fd296-11f5-4180-b82c-d234e9a173ad None None] MFA Rules not processed for user `e0c76478b6694782b896cdf9cda32d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:16:53.097621 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-eb8fd296-11f5-4180-b82c-d234e9a173ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:16:53.118246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:16:53.196494 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1215/2426] 10.222.0.73 () {58 vars in 1009 bytes} [Mon Aug 29 20:16:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:16:53.225473 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f804a-398d-4137-a1f1-cc7134d343fd service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:16:53.226964 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-842f804a-398d-4137-a1f1-cc7134d343fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:16:53.346240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f804a-398d-4137-a1f1-cc7134d343fd service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:16:53.352633 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89940a-45f6-49d7-a234-31b4aa40581c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92e30da8f394f3abc19b9e86ef126e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:16:53.353370 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89940a-45f6-49d7-a234-31b4aa40581c service swift] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:53.353739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89940a-45f6-49d7-a234-31b4aa40581c service swift] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:53.354090 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89940a-45f6-49d7-a234-31b4aa40581c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:53.365231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89940a-45f6-49d7-a234-31b4aa40581c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:16:53.367034 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89940a-45f6-49d7-a234-31b4aa40581c service swift] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:16:53.377027 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1212/2427] 10.222.0.73 () {60 vars in 1384 bytes} [Mon Aug 29 20:16:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:16:53.776366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc3892-dd76-4ca6-a4b7-65a054dbd6c3 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:53.777026 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc3892-dd76-4ca6-a4b7-65a054dbd6c3 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:53.777853 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc3892-dd76-4ca6-a4b7-65a054dbd6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:53.826208 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-17bc3892-dd76-4ca6-a4b7-65a054dbd6c3 None None] MFA Rules not processed for user `b41427ee7f3a4b478f5587b41a3b9e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:16:53.829791 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-17bc3892-dd76-4ca6-a4b7-65a054dbd6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:16:53.864382 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1216/2428] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:16:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:16:59.822517 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89940a-45f6-49d7-a234-31b4aa40581c service swift] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:16:59.824755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-2a89940a-45f6-49d7-a234-31b4aa40581c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:16:59.940112 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89940a-45f6-49d7-a234-31b4aa40581c service swift] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:16:59.961989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44f659-5e8d-420f-816d-131eafb9b11d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:16:59.962706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44f659-5e8d-420f-816d-131eafb9b11d service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:16:59.962938 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44f659-5e8d-420f-816d-131eafb9b11d service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:16:59.963210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44f659-5e8d-420f-816d-131eafb9b11d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:16:59.965321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-8b44f659-5e8d-420f-816d-131eafb9b11d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:00.111683 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44f659-5e8d-420f-816d-131eafb9b11d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:00.116524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44f659-5e8d-420f-816d-131eafb9b11d service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:00.150718 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1213/2429] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:16:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:17:03.816322 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=20 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:17:09.171905 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6181fa-b5bb-440d-94f3-bff78f60ad12 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:09.173567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6181fa-b5bb-440d-94f3-bff78f60ad12 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:09.173567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6181fa-b5bb-440d-94f3-bff78f60ad12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:09.205090 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-8c6181fa-b5bb-440d-94f3-bff78f60ad12 None None] MFA Rules not processed for user `e75b537d44524de192282b10825293f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:17:09.206856 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-8c6181fa-b5bb-440d-94f3-bff78f60ad12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:09.331207 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1217/2430] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:17:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:17:10.132839 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:17:19.312486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=25 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:17:19.918518 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d5bf3-9272-4e5f-955a-e81de4c56229 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:19.919848 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d5bf3-9272-4e5f-955a-e81de4c56229 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:19.921693 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d5bf3-9272-4e5f-955a-e81de4c56229 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:19.926415 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1214/2431] 10.222.0.73 () {54 vars in 921 bytes} [Mon Aug 29 20:17:19 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:17:19.933004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f8b92-a9ba-44fb-aeee-b09a47344b4d None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:19.933343 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f8b92-a9ba-44fb-aeee-b09a47344b4d None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:19.933554 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f8b92-a9ba-44fb-aeee-b09a47344b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:19.968157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-7a1f8b92-a9ba-44fb-aeee-b09a47344b4d None None] MFA Rules not processed for user `95cf77a92a174ff8b0503baf5c7e63be`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:17:19.970561 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-7a1f8b92-a9ba-44fb-aeee-b09a47344b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:20.076689 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1218/2432] 10.222.0.73 () {58 vars in 1018 bytes} [Mon Aug 29 20:17:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:17:20.095702 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d5bf3-9272-4e5f-955a-e81de4c56229 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:20.098005 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-294d5bf3-9272-4e5f-955a-e81de4c56229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:20.214459 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d5bf3-9272-4e5f-955a-e81de4c56229 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:20.221680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02378fa-a900-4784-a9c5-d3f22703af39 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b64b4b172467692d02851b2955b81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:20.222463 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02378fa-a900-4784-a9c5-d3f22703af39 service glance] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:20.222769 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02378fa-a900-4784-a9c5-d3f22703af39 service glance] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:20.223075 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02378fa-a900-4784-a9c5-d3f22703af39 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:20.245074 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02378fa-a900-4784-a9c5-d3f22703af39 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:20.249060 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02378fa-a900-4784-a9c5-d3f22703af39 service glance] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:20.287126 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1215/2433] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:17:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:17:20.424435 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab7548-4692-4840-ac4b-cd700c6a16b1 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:20.425282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab7548-4692-4840-ac4b-cd700c6a16b1 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:20.425282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab7548-4692-4840-ac4b-cd700c6a16b1 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:20.427616 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1219/2434] 10.222.0.73 () {54 vars in 921 bytes} [Mon Aug 29 20:17:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:17:20.441675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359d9c9-f140-4dd5-9490-115cdc6f63fb None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:20.447033 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359d9c9-f140-4dd5-9490-115cdc6f63fb None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:20.447532 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359d9c9-f140-4dd5-9490-115cdc6f63fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:20.480560 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-4359d9c9-f140-4dd5-9490-115cdc6f63fb None None] MFA Rules not processed for user `64ba58d774674ed7b233ad6416db068a`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:17:20.482852 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-4359d9c9-f140-4dd5-9490-115cdc6f63fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:20.499041 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1216/2435] 10.222.0.73 () {58 vars in 1018 bytes} [Mon Aug 29 20:17:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:17:20.520782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab7548-4692-4840-ac4b-cd700c6a16b1 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:20.524802 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-74ab7548-4692-4840-ac4b-cd700c6a16b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:20.553839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab7548-4692-4840-ac4b-cd700c6a16b1 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:20.560301 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5fe5f5-e1d3-4140-8083-d7ee804af655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:20.562538 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fe5f5-e1d3-4140-8083-d7ee804af655 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:20.563296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fe5f5-e1d3-4140-8083-d7ee804af655 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:20.563862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fe5f5-e1d3-4140-8083-d7ee804af655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:20.585955 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5fe5f5-e1d3-4140-8083-d7ee804af655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:20.585955 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5fe5f5-e1d3-4140-8083-d7ee804af655 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:20.604941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1220/2436] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:17:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:17:20.729411 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e349f-7765-4971-a92a-d348557b3c6e None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:20.730024 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e349f-7765-4971-a92a-d348557b3c6e None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:20.730515 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e349f-7765-4971-a92a-d348557b3c6e None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:20.733230 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1217/2437] 10.222.0.73 () {54 vars in 921 bytes} [Mon Aug 29 20:17:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:17:20.745373 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7a9c4-356f-47b0-9b23-f06ddc21826d None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:20.746278 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7a9c4-356f-47b0-9b23-f06ddc21826d None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:20.746860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7a9c4-356f-47b0-9b23-f06ddc21826d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:20.784554 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-81c7a9c4-356f-47b0-9b23-f06ddc21826d None None] MFA Rules not processed for user `95cf77a92a174ff8b0503baf5c7e63be`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:17:20.786434 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-81c7a9c4-356f-47b0-9b23-f06ddc21826d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:20.803518 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1221/2438] 10.222.0.73 () {58 vars in 1018 bytes} [Mon Aug 29 20:17:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:17:20.821803 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e349f-7765-4971-a92a-d348557b3c6e None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:20.846182 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e349f-7765-4971-a92a-d348557b3c6e None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:20.856675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce369597-9aec-4eca-91c8-500d73f86225 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:20.858127 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce369597-9aec-4eca-91c8-500d73f86225 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:20.858574 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce369597-9aec-4eca-91c8-500d73f86225 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:20.859060 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce369597-9aec-4eca-91c8-500d73f86225 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:20.883219 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce369597-9aec-4eca-91c8-500d73f86225 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:20.885862 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce369597-9aec-4eca-91c8-500d73f86225 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:20.938681 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1218/2439] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:17:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:17:22.012293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7a9c4-356f-47b0-9b23-f06ddc21826d None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:22.015204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-81c7a9c4-356f-47b0-9b23-f06ddc21826d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:22.119706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7a9c4-356f-47b0-9b23-f06ddc21826d None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:22.128247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941eaa8-ef07-44dc-b1e1-15998cbd8d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:22.128604 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941eaa8-ef07-44dc-b1e1-15998cbd8d98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:22.128883 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941eaa8-ef07-44dc-b1e1-15998cbd8d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:22.129297 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941eaa8-ef07-44dc-b1e1-15998cbd8d98 admin admin] PATH_INFO: `/v3/users/542eb802a8584c0a9ab1edd3e8fa41ee` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:22.131395 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941eaa8-ef07-44dc-b1e1-15998cbd8d98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=542eb802a8584c0a9ab1edd3e8fa41ee)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:22.134615 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941eaa8-ef07-44dc-b1e1-15998cbd8d98 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:22.273079 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e941eaa8-ef07-44dc-b1e1-15998cbd8d98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '542eb802a8584c0a9ab1edd3e8fa41ee'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:17:22.338695 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e941eaa8-ef07-44dc-b1e1-15998cbd8d98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '542eb802a8584c0a9ab1edd3e8fa41ee'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:17:22.361578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e941eaa8-ef07-44dc-b1e1-15998cbd8d98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '542eb802a8584c0a9ab1edd3e8fa41ee'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:17:22.370350 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:17:22.394702 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e941eaa8-ef07-44dc-b1e1-15998cbd8d98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '542eb802a8584c0a9ab1edd3e8fa41ee'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:17:22.395403 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e941eaa8-ef07-44dc-b1e1-15998cbd8d98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '542eb802a8584c0a9ab1edd3e8fa41ee'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:17:22.413268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1222/2440] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:17:22 2022] DELETE /identity/v3/users/542eb802a8584c0a9ab1edd3e8fa41ee => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:17:22.415786 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f79308-12e3-4fe7-926b-d0534e36539b None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:22.416472 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f79308-12e3-4fe7-926b-d0534e36539b None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:22.416472 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f79308-12e3-4fe7-926b-d0534e36539b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:22.460006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-06f79308-12e3-4fe7-926b-d0534e36539b None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:17:22.464778 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-06f79308-12e3-4fe7-926b-d0534e36539b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:22.613123 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1219/2441] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:17:22.629161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941eaa8-ef07-44dc-b1e1-15998cbd8d98 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:22.631248 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-e941eaa8-ef07-44dc-b1e1-15998cbd8d98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:22.766060 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941eaa8-ef07-44dc-b1e1-15998cbd8d98 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:22.774349 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6482863-0575-4a84-b335-2b8c561deae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:22.775390 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6482863-0575-4a84-b335-2b8c561deae0 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:22.775821 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6482863-0575-4a84-b335-2b8c561deae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:22.776304 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6482863-0575-4a84-b335-2b8c561deae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:22.820140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6482863-0575-4a84-b335-2b8c561deae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:22.821831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6482863-0575-4a84-b335-2b8c561deae0 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:22.839677 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1223/2442] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:17:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:17:23.515952 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f79308-12e3-4fe7-926b-d0534e36539b None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:23.519103 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-06f79308-12e3-4fe7-926b-d0534e36539b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:23.544767 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f79308-12e3-4fe7-926b-d0534e36539b None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:23.553904 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da476ba-3c9b-4a5b-b14f-b787b5269caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:23.554978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da476ba-3c9b-4a5b-b14f-b787b5269caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:23.555502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da476ba-3c9b-4a5b-b14f-b787b5269caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:23.556024 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da476ba-3c9b-4a5b-b14f-b787b5269caf admin admin] PATH_INFO: `/v3/projects/58302c93bbd949569c1bd3e758a023b8` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:23.557640 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da476ba-3c9b-4a5b-b14f-b787b5269caf admin admin] RBAC: Authorizing `identity:delete_project(project_id=58302c93bbd949569c1bd3e758a023b8)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:23.562416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da476ba-3c9b-4a5b-b14f-b787b5269caf admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:23.659919 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-0da476ba-3c9b-4a5b-b14f-b787b5269caf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58302c93bbd949569c1bd3e758a023b8', 'request_id': 'req-0da476ba-3c9b-4a5b-b14f-b787b5269caf'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:17:23.670625 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-0da476ba-3c9b-4a5b-b14f-b787b5269caf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58302c93bbd949569c1bd3e758a023b8', 'request_id': 'req-0da476ba-3c9b-4a5b-b14f-b787b5269caf'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:17:23.681518 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-0da476ba-3c9b-4a5b-b14f-b787b5269caf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58302c93bbd949569c1bd3e758a023b8', 'request_id': 'req-0da476ba-3c9b-4a5b-b14f-b787b5269caf'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:17:23.704822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-0da476ba-3c9b-4a5b-b14f-b787b5269caf admin admin] The token cache is being invalidate because project 58302c93bbd949569c1bd3e758a023b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:17:23.705345 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-0da476ba-3c9b-4a5b-b14f-b787b5269caf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:17:23.706566 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-0da476ba-3c9b-4a5b-b14f-b787b5269caf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:17:23.709575 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1220/2443] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:17:23 2022] DELETE /identity/v3/projects/58302c93bbd949569c1bd3e758a023b8 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:17:23.715606 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970013c-9dc5-46ab-b6fe-ea7ccc42265b None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:23.717057 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970013c-9dc5-46ab-b6fe-ea7ccc42265b None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:23.718009 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970013c-9dc5-46ab-b6fe-ea7ccc42265b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:23.754036 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-0970013c-9dc5-46ab-b6fe-ea7ccc42265b None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:17:23.755976 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-0970013c-9dc5-46ab-b6fe-ea7ccc42265b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:23.889224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1224/2444] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:17:23.899383 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc280661-e1eb-4ea7-a043-73631555b2f9 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:23.899659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc280661-e1eb-4ea7-a043-73631555b2f9 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:23.899920 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc280661-e1eb-4ea7-a043-73631555b2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:23.918327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:17:23.925389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-bc280661-e1eb-4ea7-a043-73631555b2f9 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:17:23.927871 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-bc280661-e1eb-4ea7-a043-73631555b2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:23.943037 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1221/2445] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:17:23.947926 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970013c-9dc5-46ab-b6fe-ea7ccc42265b None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:23.959486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970013c-9dc5-46ab-b6fe-ea7ccc42265b None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:23.964462 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04025f80-abdc-49d9-bdb3-4c8bea2b966d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:23.965089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04025f80-abdc-49d9-bdb3-4c8bea2b966d admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:23.965562 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04025f80-abdc-49d9-bdb3-4c8bea2b966d admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:23.966167 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04025f80-abdc-49d9-bdb3-4c8bea2b966d admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:23.967005 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04025f80-abdc-49d9-bdb3-4c8bea2b966d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:17:23.967468 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04025f80-abdc-49d9-bdb3-4c8bea2b966d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:23.970013 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04025f80-abdc-49d9-bdb3-4c8bea2b966d admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:23.990721 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1225/2446] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:17:23.997748 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc280661-e1eb-4ea7-a043-73631555b2f9 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:24.013465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc280661-e1eb-4ea7-a043-73631555b2f9 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:24.021139 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93aa730-adf1-47e7-873f-76befa573f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:24.023096 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93aa730-adf1-47e7-873f-76befa573f74 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:24.023622 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93aa730-adf1-47e7-873f-76befa573f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:24.024150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93aa730-adf1-47e7-873f-76befa573f74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:24.025065 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93aa730-adf1-47e7-873f-76befa573f74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:24.028498 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93aa730-adf1-47e7-873f-76befa573f74 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:24.093842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1222/2447] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:17:23 2022] POST /identity/v3/projects => generated 391 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:17:24.098809 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04025f80-abdc-49d9-bdb3-4c8bea2b966d admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:24.135105 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04025f80-abdc-49d9-bdb3-4c8bea2b966d admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:24.264962 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34cb19-f54b-4a7c-8094-dd1280409b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:24.265970 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34cb19-f54b-4a7c-8094-dd1280409b56 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:24.267214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34cb19-f54b-4a7c-8094-dd1280409b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:24.267480 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34cb19-f54b-4a7c-8094-dd1280409b56 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:24.268238 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca34cb19-f54b-4a7c-8094-dd1280409b56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:24.270539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca34cb19-f54b-4a7c-8094-dd1280409b56 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:24.320876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1226/2448] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:17:24 2022] POST /identity/v3/users => generated 296 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:17:24.326718 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93aa730-adf1-47e7-873f-76befa573f74 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:24.352759 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93aa730-adf1-47e7-873f-76befa573f74 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:24.363084 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130409b-6912-47b2-805d-f3e773bb8ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:24.364619 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130409b-6912-47b2-805d-f3e773bb8ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:24.365650 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130409b-6912-47b2-805d-f3e773bb8ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:24.366825 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130409b-6912-47b2-805d-f3e773bb8ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:24.367888 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130409b-6912-47b2-805d-f3e773bb8ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:24.371136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130409b-6912-47b2-805d-f3e773bb8ca7 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:24.390941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1223/2449] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:17:24.398205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34cb19-f54b-4a7c-8094-dd1280409b56 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:24.414235 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34cb19-f54b-4a7c-8094-dd1280409b56 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:24.422130 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f80f7a-83e7-446c-946e-c5e3ecea0094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:24.423095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f80f7a-83e7-446c-946e-c5e3ecea0094 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:24.423384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f80f7a-83e7-446c-946e-c5e3ecea0094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:24.423798 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f80f7a-83e7-446c-946e-c5e3ecea0094 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:24.424484 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f80f7a-83e7-446c-946e-c5e3ecea0094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:24.426731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f80f7a-83e7-446c-946e-c5e3ecea0094 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:24.437000 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:17:24.613915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1227/2450] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:17:24.618691 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130409b-6912-47b2-805d-f3e773bb8ca7 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:24.635071 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130409b-6912-47b2-805d-f3e773bb8ca7 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:24.642773 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0789338c-b2f0-427f-9665-54b666135a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:24.643763 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789338c-b2f0-427f-9665-54b666135a44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:24.644198 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789338c-b2f0-427f-9665-54b666135a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:24.644665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789338c-b2f0-427f-9665-54b666135a44 admin admin] PATH_INFO: `/v3/projects/e2d9861d82884e048e31f8c740ec4660/users/05fe5db33613480a93bdbcc7650b53e7/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:24.667601 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0789338c-b2f0-427f-9665-54b666135a44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2d9861d82884e048e31f8c740ec4660, user_id=05fe5db33613480a93bdbcc7650b53e7, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:24.670559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0789338c-b2f0-427f-9665-54b666135a44 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:24.701961 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1224/2451] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:17:24 2022] PUT /identity/v3/projects/e2d9861d82884e048e31f8c740ec4660/users/05fe5db33613480a93bdbcc7650b53e7/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:17:24.715853 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bbe0a-f36c-4533-8454-d48aab23b8b5 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:24.716640 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bbe0a-f36c-4533-8454-d48aab23b8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:24.716786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bbe0a-f36c-4533-8454-d48aab23b8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:24.752923 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-5d1bbe0a-f36c-4533-8454-d48aab23b8b5 None None] MFA Rules not processed for user `05fe5db33613480a93bdbcc7650b53e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:17:24.754296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-5d1bbe0a-f36c-4533-8454-d48aab23b8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:24.859528 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1228/2452] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:17:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:17:24.879168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0789338c-b2f0-427f-9665-54b666135a44 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:24.881862 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-0789338c-b2f0-427f-9665-54b666135a44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:24.970359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0789338c-b2f0-427f-9665-54b666135a44 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:24.974852 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cbf5eb-53e9-4041-bfd8-9daebe26480f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:24.975476 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbf5eb-53e9-4041-bfd8-9daebe26480f service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:24.975748 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbf5eb-53e9-4041-bfd8-9daebe26480f service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:24.976015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbf5eb-53e9-4041-bfd8-9daebe26480f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:24.987358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cbf5eb-53e9-4041-bfd8-9daebe26480f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:24.989261 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cbf5eb-53e9-4041-bfd8-9daebe26480f service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:25.004217 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1225/2453] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:17:24 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:17:31.175724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1bbe0a-f36c-4533-8454-d48aab23b8b5 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:31.177302 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-5d1bbe0a-f36c-4533-8454-d48aab23b8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:31.291514 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1bbe0a-f36c-4533-8454-d48aab23b8b5 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:31.296700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243c538-3c1a-4a77-95e4-a6165a98b31a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:31.298413 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794fc06-dba6-4997-9192-252f3339b205 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:31.299143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243c538-3c1a-4a77-95e4-a6165a98b31a service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:31.299143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243c538-3c1a-4a77-95e4-a6165a98b31a service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:31.299473 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794fc06-dba6-4997-9192-252f3339b205 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:31.299634 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243c538-3c1a-4a77-95e4-a6165a98b31a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:31.299803 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794fc06-dba6-4997-9192-252f3339b205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:31.302080 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-9243c538-3c1a-4a77-95e4-a6165a98b31a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:31.324139 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-5794fc06-dba6-4997-9192-252f3339b205 None None] MFA Rules not processed for user `e0c76478b6694782b896cdf9cda32d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:17:31.325768 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-5794fc06-dba6-4997-9192-252f3339b205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:31.429128 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1226/2454] 10.222.0.73 () {58 vars in 1009 bytes} [Mon Aug 29 20:17:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:17:31.438033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243c538-3c1a-4a77-95e4-a6165a98b31a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:31.441392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243c538-3c1a-4a77-95e4-a6165a98b31a service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:31.448017 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5794fc06-dba6-4997-9192-252f3339b205 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:31.450445 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-5794fc06-dba6-4997-9192-252f3339b205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:31.467826 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1229/2455] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:17:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:17:31.545287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:17:31.579816 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5794fc06-dba6-4997-9192-252f3339b205 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:31.589005 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167f8d6-ff9c-4d01-8f37-cb2a645f18c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92e30da8f394f3abc19b9e86ef126e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:31.589885 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167f8d6-ff9c-4d01-8f37-cb2a645f18c6 service swift] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:31.590316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167f8d6-ff9c-4d01-8f37-cb2a645f18c6 service swift] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:31.590655 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167f8d6-ff9c-4d01-8f37-cb2a645f18c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:31.602414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167f8d6-ff9c-4d01-8f37-cb2a645f18c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:31.604187 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167f8d6-ff9c-4d01-8f37-cb2a645f18c6 service swift] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:31.614990 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1227/2456] 10.222.0.73 () {60 vars in 1384 bytes} [Mon Aug 29 20:17:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 20:17:32.073427 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243c538-3c1a-4a77-95e4-a6165a98b31a service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:32.075688 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-9243c538-3c1a-4a77-95e4-a6165a98b31a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:32.110855 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243c538-3c1a-4a77-95e4-a6165a98b31a service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:32.118521 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee6d52-8bfc-43f5-a79f-4dd65d9035e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:32.119482 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee6d52-8bfc-43f5-a79f-4dd65d9035e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:32.119931 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee6d52-8bfc-43f5-a79f-4dd65d9035e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:32.120370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee6d52-8bfc-43f5-a79f-4dd65d9035e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:32.123104 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-7cee6d52-8bfc-43f5-a79f-4dd65d9035e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:32.139795 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee6d52-8bfc-43f5-a79f-4dd65d9035e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:32.142505 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee6d52-8bfc-43f5-a79f-4dd65d9035e1 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:32.162034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1230/2457] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:17:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:17:41.609990 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=9 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:17:42.151205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=92 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:17:42.568586 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167f8d6-ff9c-4d01-8f37-cb2a645f18c6 service swift] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:42.572560 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-b167f8d6-ff9c-4d01-8f37-cb2a645f18c6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:42.599092 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167f8d6-ff9c-4d01-8f37-cb2a645f18c6 service swift] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:42.606029 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa68b7-f6eb-4a4c-834c-96145462ad87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:42.606775 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa68b7-f6eb-4a4c-834c-96145462ad87 service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:42.607055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa68b7-f6eb-4a4c-834c-96145462ad87 service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:42.607660 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa68b7-f6eb-4a4c-834c-96145462ad87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:42.609754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-dbaa68b7-f6eb-4a4c-834c-96145462ad87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:42.732886 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa68b7-f6eb-4a4c-834c-96145462ad87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:42.738172 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa68b7-f6eb-4a4c-834c-96145462ad87 service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:42.763395 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1228/2458] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:17:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:17:46.983458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee6d52-8bfc-43f5-a79f-4dd65d9035e1 service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:47.077381 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee6d52-8bfc-43f5-a79f-4dd65d9035e1 service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:47.082441 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01594e5-1da0-44b1-9c5b-a329cb7c12a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:47.083159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01594e5-1da0-44b1-9c5b-a329cb7c12a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:47.083627 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01594e5-1da0-44b1-9c5b-a329cb7c12a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:47.083912 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01594e5-1da0-44b1-9c5b-a329cb7c12a6 admin admin] PATH_INFO: `/v3/users/05fe5db33613480a93bdbcc7650b53e7` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:47.085144 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01594e5-1da0-44b1-9c5b-a329cb7c12a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05fe5db33613480a93bdbcc7650b53e7)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:47.087163 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01594e5-1da0-44b1-9c5b-a329cb7c12a6 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:47.177416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d01594e5-1da0-44b1-9c5b-a329cb7c12a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05fe5db33613480a93bdbcc7650b53e7'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:17:47.194273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d01594e5-1da0-44b1-9c5b-a329cb7c12a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05fe5db33613480a93bdbcc7650b53e7'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:17:47.204902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d01594e5-1da0-44b1-9c5b-a329cb7c12a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05fe5db33613480a93bdbcc7650b53e7'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:17:47.215562 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d01594e5-1da0-44b1-9c5b-a329cb7c12a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05fe5db33613480a93bdbcc7650b53e7'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:17:47.216394 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d01594e5-1da0-44b1-9c5b-a329cb7c12a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05fe5db33613480a93bdbcc7650b53e7'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:17:47.228453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1231/2459] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:17:46 2022] DELETE /identity/v3/users/05fe5db33613480a93bdbcc7650b53e7 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:17:47.234446 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb8113-3e5e-49fe-94e2-38fcfed6bf39 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:47.235202 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb8113-3e5e-49fe-94e2-38fcfed6bf39 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:47.235486 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb8113-3e5e-49fe-94e2-38fcfed6bf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:47.258640 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-2bcb8113-3e5e-49fe-94e2-38fcfed6bf39 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:17:47.259894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-2bcb8113-3e5e-49fe-94e2-38fcfed6bf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:47.332949 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1229/2460] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:17:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:17:47.343887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01594e5-1da0-44b1-9c5b-a329cb7c12a6 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:47.346111 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-d01594e5-1da0-44b1-9c5b-a329cb7c12a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:47.412869 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01594e5-1da0-44b1-9c5b-a329cb7c12a6 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:47.416642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5f365-2db7-40dd-bb9c-791eb9b21ccf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:47.417251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5f365-2db7-40dd-bb9c-791eb9b21ccf service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:47.417522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5f365-2db7-40dd-bb9c-791eb9b21ccf service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:47.417820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5f365-2db7-40dd-bb9c-791eb9b21ccf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:47.428884 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b5f365-2db7-40dd-bb9c-791eb9b21ccf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:47.430388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b5f365-2db7-40dd-bb9c-791eb9b21ccf service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:47.445515 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1232/2461] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:17:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:17:47.813752 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb8113-3e5e-49fe-94e2-38fcfed6bf39 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:47.815883 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-2bcb8113-3e5e-49fe-94e2-38fcfed6bf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:47.832067 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb8113-3e5e-49fe-94e2-38fcfed6bf39 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:47.839467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68458028-6963-4f5d-8db9-a6195fa6205d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:47.841293 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68458028-6963-4f5d-8db9-a6195fa6205d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:47.841784 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68458028-6963-4f5d-8db9-a6195fa6205d admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:47.842322 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68458028-6963-4f5d-8db9-a6195fa6205d admin admin] PATH_INFO: `/v3/projects/e2d9861d82884e048e31f8c740ec4660` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:47.844174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68458028-6963-4f5d-8db9-a6195fa6205d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2d9861d82884e048e31f8c740ec4660)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:47.849567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68458028-6963-4f5d-8db9-a6195fa6205d admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:47.947784 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-68458028-6963-4f5d-8db9-a6195fa6205d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2d9861d82884e048e31f8c740ec4660', 'request_id': 'req-68458028-6963-4f5d-8db9-a6195fa6205d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:17:47.961496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-68458028-6963-4f5d-8db9-a6195fa6205d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2d9861d82884e048e31f8c740ec4660', 'request_id': 'req-68458028-6963-4f5d-8db9-a6195fa6205d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:17:47.971517 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-68458028-6963-4f5d-8db9-a6195fa6205d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2d9861d82884e048e31f8c740ec4660', 'request_id': 'req-68458028-6963-4f5d-8db9-a6195fa6205d'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:17:47.991889 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-68458028-6963-4f5d-8db9-a6195fa6205d admin admin] The token cache is being invalidate because project e2d9861d82884e048e31f8c740ec4660 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:17:47.992341 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-68458028-6963-4f5d-8db9-a6195fa6205d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:17:47.993117 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-68458028-6963-4f5d-8db9-a6195fa6205d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:17:47.995054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1230/2462] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:17:47 2022] DELETE /identity/v3/projects/e2d9861d82884e048e31f8c740ec4660 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:17:48.000841 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311bcd2-8319-45b4-a960-0eb8260e561b None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:48.002320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311bcd2-8319-45b4-a960-0eb8260e561b None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:48.002592 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311bcd2-8319-45b4-a960-0eb8260e561b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:48.026132 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-8311bcd2-8319-45b4-a960-0eb8260e561b None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:17:48.027372 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-8311bcd2-8319-45b4-a960-0eb8260e561b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:48.052876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:17:48.128215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1233/2463] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:17:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:17:48.142927 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f23c1-d236-47f3-ae09-746c9b7a50e7 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:48.143234 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f23c1-d236-47f3-ae09-746c9b7a50e7 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:48.143496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f23c1-d236-47f3-ae09-746c9b7a50e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:48.166201 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-250f23c1-d236-47f3-ae09-746c9b7a50e7 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:17:48.167603 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-250f23c1-d236-47f3-ae09-746c9b7a50e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:48.176533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1231/2464] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:17:48.181025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311bcd2-8319-45b4-a960-0eb8260e561b None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:48.201828 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311bcd2-8319-45b4-a960-0eb8260e561b None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:48.209424 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d121636-b5ae-4608-bc3f-1a5f6bd36f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:48.210413 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d121636-b5ae-4608-bc3f-1a5f6bd36f98 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:48.210807 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d121636-b5ae-4608-bc3f-1a5f6bd36f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:48.211286 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d121636-b5ae-4608-bc3f-1a5f6bd36f98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:48.212552 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d121636-b5ae-4608-bc3f-1a5f6bd36f98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:17:48.212973 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d121636-b5ae-4608-bc3f-1a5f6bd36f98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:48.215708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d121636-b5ae-4608-bc3f-1a5f6bd36f98 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:48.234747 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1234/2465] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:17:48.241243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f23c1-d236-47f3-ae09-746c9b7a50e7 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:48.251269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f23c1-d236-47f3-ae09-746c9b7a50e7 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:48.256296 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce23bc-9fef-4da8-85b1-6c90fb6e23c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:48.256906 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce23bc-9fef-4da8-85b1-6c90fb6e23c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:48.257186 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce23bc-9fef-4da8-85b1-6c90fb6e23c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:48.257461 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce23bc-9fef-4da8-85b1-6c90fb6e23c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:48.258154 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bce23bc-9fef-4da8-85b1-6c90fb6e23c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:48.260064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bce23bc-9fef-4da8-85b1-6c90fb6e23c6 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:48.276460 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:17:48.295632 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1232/2466] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:17:48 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:17:48.300892 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d121636-b5ae-4608-bc3f-1a5f6bd36f98 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:48.312959 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d121636-b5ae-4608-bc3f-1a5f6bd36f98 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:48.406781 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252981c3-4365-4c17-ad83-95857c691c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:48.408133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252981c3-4365-4c17-ad83-95857c691c25 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:48.408364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252981c3-4365-4c17-ad83-95857c691c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:48.408846 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252981c3-4365-4c17-ad83-95857c691c25 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:48.409912 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252981c3-4365-4c17-ad83-95857c691c25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:48.411942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252981c3-4365-4c17-ad83-95857c691c25 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:48.439578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1235/2467] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:17:48 2022] POST /identity/v3/users => generated 290 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:17:48.444123 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce23bc-9fef-4da8-85b1-6c90fb6e23c6 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:48.456317 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce23bc-9fef-4da8-85b1-6c90fb6e23c6 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:48.461671 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce502fcb-1f03-4f3d-8d6e-22915cfa60af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:48.462449 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce502fcb-1f03-4f3d-8d6e-22915cfa60af admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:48.462828 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce502fcb-1f03-4f3d-8d6e-22915cfa60af admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:48.463096 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce502fcb-1f03-4f3d-8d6e-22915cfa60af admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:48.463901 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce502fcb-1f03-4f3d-8d6e-22915cfa60af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:48.465962 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce502fcb-1f03-4f3d-8d6e-22915cfa60af admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:48.478898 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1233/2468] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:17:48.484286 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252981c3-4365-4c17-ad83-95857c691c25 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:48.502259 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252981c3-4365-4c17-ad83-95857c691c25 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:48.509758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a814f2-47ea-4347-a670-277792cb39db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:48.510721 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a814f2-47ea-4347-a670-277792cb39db admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:48.511203 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a814f2-47ea-4347-a670-277792cb39db admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:48.511636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a814f2-47ea-4347-a670-277792cb39db admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:48.512319 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a814f2-47ea-4347-a670-277792cb39db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:48.513957 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a814f2-47ea-4347-a670-277792cb39db admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:48.535550 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1236/2469] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:17:48.540208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce502fcb-1f03-4f3d-8d6e-22915cfa60af admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:48.551616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce502fcb-1f03-4f3d-8d6e-22915cfa60af admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:48.556416 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cca6a-521b-46db-a459-8985954615c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:48.557072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cca6a-521b-46db-a459-8985954615c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:48.557353 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cca6a-521b-46db-a459-8985954615c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:48.557624 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cca6a-521b-46db-a459-8985954615c5 admin admin] PATH_INFO: `/v3/projects/ff3969ffe8ff411b8773e1f3f3cb0b40/users/8640600ca0d745cda547a1027c947089/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:48.572989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79cca6a-521b-46db-a459-8985954615c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff3969ffe8ff411b8773e1f3f3cb0b40, user_id=8640600ca0d745cda547a1027c947089, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:48.575712 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79cca6a-521b-46db-a459-8985954615c5 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:48.594952 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1234/2470] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:17:48 2022] PUT /identity/v3/projects/ff3969ffe8ff411b8773e1f3f3cb0b40/users/8640600ca0d745cda547a1027c947089/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:17:48.601171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed0a4f-b0ce-467f-9695-75338524b766 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:48.601448 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed0a4f-b0ce-467f-9695-75338524b766 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:48.601731 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed0a4f-b0ce-467f-9695-75338524b766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:48.627401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-d9ed0a4f-b0ce-467f-9695-75338524b766 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:17:48.629275 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-d9ed0a4f-b0ce-467f-9695-75338524b766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:48.700598 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1237/2471] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:17:48.711303 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cca6a-521b-46db-a459-8985954615c5 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:48.713634 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-a79cca6a-521b-46db-a459-8985954615c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:48.793008 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cca6a-521b-46db-a459-8985954615c5 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:48.800691 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a9bf8-0f2c-47f2-9f20-6e1a260b459a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:48.801688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a9bf8-0f2c-47f2-9f20-6e1a260b459a service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:48.802273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a9bf8-0f2c-47f2-9f20-6e1a260b459a service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:48.802777 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a9bf8-0f2c-47f2-9f20-6e1a260b459a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:48.815180 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1a9bf8-0f2c-47f2-9f20-6e1a260b459a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:48.817025 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1a9bf8-0f2c-47f2-9f20-6e1a260b459a service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:48.831045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1235/2472] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:17:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:17:56.025039 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02acd1-8a73-47f1-b2c0-4d2f5e6de1f6 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:56.026354 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02acd1-8a73-47f1-b2c0-4d2f5e6de1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:56.027536 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02acd1-8a73-47f1-b2c0-4d2f5e6de1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:56.053100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-ad02acd1-8a73-47f1-b2c0-4d2f5e6de1f6 None None] MFA Rules not processed for user `e0c76478b6694782b896cdf9cda32d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:17:56.054636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ad02acd1-8a73-47f1-b2c0-4d2f5e6de1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:56.086001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:17:56.153817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1238/2473] 10.222.0.73 () {58 vars in 1009 bytes} [Mon Aug 29 20:17:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:17:56.210287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a9bf8-0f2c-47f2-9f20-6e1a260b459a service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:56.211886 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-fa1a9bf8-0f2c-47f2-9f20-6e1a260b459a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:56.311946 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a9bf8-0f2c-47f2-9f20-6e1a260b459a service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:56.320239 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d7187-b80b-4c96-9f7e-24e8f54c09b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92e30da8f394f3abc19b9e86ef126e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:56.320935 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d7187-b80b-4c96-9f7e-24e8f54c09b5 service swift] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:56.321224 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d7187-b80b-4c96-9f7e-24e8f54c09b5 service swift] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:56.321552 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d7187-b80b-4c96-9f7e-24e8f54c09b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:56.333436 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d7187-b80b-4c96-9f7e-24e8f54c09b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:56.335163 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d7187-b80b-4c96-9f7e-24e8f54c09b5 service swift] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:56.344675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1236/2474] 10.222.0.73 () {60 vars in 1384 bytes} [Mon Aug 29 20:17:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:17:57.141633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe256c9-f5c7-4c1e-bed2-d9b1f1f9722b None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:57.143064 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe256c9-f5c7-4c1e-bed2-d9b1f1f9722b None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:57.143064 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe256c9-f5c7-4c1e-bed2-d9b1f1f9722b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:57.182206 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-0fe256c9-f5c7-4c1e-bed2-d9b1f1f9722b None None] MFA Rules not processed for user `8640600ca0d745cda547a1027c947089`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:17:57.184049 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-0fe256c9-f5c7-4c1e-bed2-d9b1f1f9722b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:57.238368 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7005ca-d471-4d58-9094-b1380e4eee07 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:57.238927 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7005ca-d471-4d58-9094-b1380e4eee07 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:57.239068 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7005ca-d471-4d58-9094-b1380e4eee07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:57.262707 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-5c7005ca-d471-4d58-9094-b1380e4eee07 None None] MFA Rules not processed for user `e0c76478b6694782b896cdf9cda32d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:17:57.263940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-5c7005ca-d471-4d58-9094-b1380e4eee07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:57.272495 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1237/2476] 10.222.0.73 () {58 vars in 967 bytes} [Mon Aug 29 20:17:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:17:57.272700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1239/2476] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:17:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:17:57.278442 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe256c9-f5c7-4c1e-bed2-d9b1f1f9722b None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:57.287330 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7005ca-d471-4d58-9094-b1380e4eee07 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:57.293572 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe256c9-f5c7-4c1e-bed2-d9b1f1f9722b None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:57.301155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd5e5c-6aa9-45b0-a2ae-c1a26de389ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:57.302138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd5e5c-6aa9-45b0-a2ae-c1a26de389ee admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:57.302436 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd5e5c-6aa9-45b0-a2ae-c1a26de389ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:57.302755 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd5e5c-6aa9-45b0-a2ae-c1a26de389ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:57.303835 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd5e5c-6aa9-45b0-a2ae-c1a26de389ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:57.305299 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7005ca-d471-4d58-9094-b1380e4eee07 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:57.307247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd5e5c-6aa9-45b0-a2ae-c1a26de389ee admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:57.310837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209161b3-3d1d-4fcb-b7a0-757c95ed0b7d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92e30da8f394f3abc19b9e86ef126e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:57.311458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209161b3-3d1d-4fcb-b7a0-757c95ed0b7d service swift] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:57.311739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209161b3-3d1d-4fcb-b7a0-757c95ed0b7d service swift] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:57.312017 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209161b3-3d1d-4fcb-b7a0-757c95ed0b7d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:57.325375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209161b3-3d1d-4fcb-b7a0-757c95ed0b7d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:57.327056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209161b3-3d1d-4fcb-b7a0-757c95ed0b7d service swift] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:57.337276 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1238/2477] 10.222.0.73 () {60 vars in 1384 bytes} [Mon Aug 29 20:17:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 20:17:57.355391 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1240/2478] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:17:57 2022] POST /identity/v3/projects => generated 379 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:17:57.359117 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209161b3-3d1d-4fcb-b7a0-757c95ed0b7d service swift] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:57.385120 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209161b3-3d1d-4fcb-b7a0-757c95ed0b7d service swift] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:57.441015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:17:57.513052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20502ba-23ca-4c67-9cce-ba70a3cf6231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:57.514160 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20502ba-23ca-4c67-9cce-ba70a3cf6231 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:57.516190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20502ba-23ca-4c67-9cce-ba70a3cf6231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:57.517091 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20502ba-23ca-4c67-9cce-ba70a3cf6231 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:57.518148 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20502ba-23ca-4c67-9cce-ba70a3cf6231 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:57.520795 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20502ba-23ca-4c67-9cce-ba70a3cf6231 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:57.555195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1239/2479] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:17:57 2022] POST /identity/v3/users => generated 296 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:17:57.557778 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd5e5c-6aa9-45b0-a2ae-c1a26de389ee admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:57.576825 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd5e5c-6aa9-45b0-a2ae-c1a26de389ee admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:57.591834 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca52f6-f76f-4ac7-bfa4-c91bdf60fcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:57.593148 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca52f6-f76f-4ac7-bfa4-c91bdf60fcfa admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:57.593464 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca52f6-f76f-4ac7-bfa4-c91bdf60fcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:57.594903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca52f6-f76f-4ac7-bfa4-c91bdf60fcfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:57.596825 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca52f6-f76f-4ac7-bfa4-c91bdf60fcfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:57.602288 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca52f6-f76f-4ac7-bfa4-c91bdf60fcfa admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:57.654847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1241/2480] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:17:57.656769 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20502ba-23ca-4c67-9cce-ba70a3cf6231 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:57.679220 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20502ba-23ca-4c67-9cce-ba70a3cf6231 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:57.697159 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589452d-5a3b-462c-b3de-facd976ebb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:57.697631 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589452d-5a3b-462c-b3de-facd976ebb4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:57.698075 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589452d-5a3b-462c-b3de-facd976ebb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:57.698476 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589452d-5a3b-462c-b3de-facd976ebb4d admin admin] PATH_INFO: `/v3/projects/a70786727b784db6972c1f3f1433dc36/users/a73812ad386243e29fc41ef97037ae85/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:57.756566 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f589452d-5a3b-462c-b3de-facd976ebb4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a70786727b784db6972c1f3f1433dc36, user_id=a73812ad386243e29fc41ef97037ae85, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:57.758805 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f589452d-5a3b-462c-b3de-facd976ebb4d admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:57.795362 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1240/2481] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:17:57 2022] PUT /identity/v3/projects/a70786727b784db6972c1f3f1433dc36/users/a73812ad386243e29fc41ef97037ae85/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:17:57.801241 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca52f6-f76f-4ac7-bfa4-c91bdf60fcfa admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:57.802343 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a76c0-6615-4212-9846-9dc6d312ae16 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:57.802641 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a76c0-6615-4212-9846-9dc6d312ae16 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:57.802920 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a76c0-6615-4212-9846-9dc6d312ae16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:57.827748 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca52f6-f76f-4ac7-bfa4-c91bdf60fcfa admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:57.836399 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-ae5a76c0-6615-4212-9846-9dc6d312ae16 None None] MFA Rules not processed for user `e0c76478b6694782b896cdf9cda32d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:17:57.837803 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-ae5a76c0-6615-4212-9846-9dc6d312ae16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:57.953515 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1241/2482] 10.222.0.73 () {58 vars in 1009 bytes} [Mon Aug 29 20:17:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:17:57.964875 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a76c0-6615-4212-9846-9dc6d312ae16 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:57.967197 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b392b-dfa6-4786-9598-cba2ac8fd610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:57.967827 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b392b-dfa6-4786-9598-cba2ac8fd610 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:57.968106 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b392b-dfa6-4786-9598-cba2ac8fd610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:57.968382 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b392b-dfa6-4786-9598-cba2ac8fd610 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:57.969038 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b392b-dfa6-4786-9598-cba2ac8fd610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:57.972591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b392b-dfa6-4786-9598-cba2ac8fd610 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:58.010055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1242/2483] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:17:58.011371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b392b-dfa6-4786-9598-cba2ac8fd610 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:58.025279 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b392b-dfa6-4786-9598-cba2ac8fd610 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:58.030728 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa33581-1dc0-4021-8259-14985705db3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:58.031415 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa33581-1dc0-4021-8259-14985705db3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:58.031695 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa33581-1dc0-4021-8259-14985705db3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:58.032023 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa33581-1dc0-4021-8259-14985705db3c admin admin] PATH_INFO: `/v3/projects/a70786727b784db6972c1f3f1433dc36/users/a73812ad386243e29fc41ef97037ae85/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:58.034613 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa33581-1dc0-4021-8259-14985705db3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a70786727b784db6972c1f3f1433dc36, user_id=a73812ad386243e29fc41ef97037ae85, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:58.037657 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa33581-1dc0-4021-8259-14985705db3c admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:58.055837 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1243/2484] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:17:58 2022] PUT /identity/v3/projects/a70786727b784db6972c1f3f1433dc36/users/a73812ad386243e29fc41ef97037ae85/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:17:58.115522 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a76c0-6615-4212-9846-9dc6d312ae16 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:58.186363 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:17:58.224669 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b70782-32e2-4a47-9f01-9d0ee788718d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92e30da8f394f3abc19b9e86ef126e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:58.225431 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b70782-32e2-4a47-9f01-9d0ee788718d service swift] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:58.225921 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b70782-32e2-4a47-9f01-9d0ee788718d service swift] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:58.226225 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b70782-32e2-4a47-9f01-9d0ee788718d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:58.314527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b70782-32e2-4a47-9f01-9d0ee788718d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:58.317176 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b70782-32e2-4a47-9f01-9d0ee788718d service swift] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:58.344789 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1242/2485] 10.222.0.73 () {60 vars in 1384 bytes} [Mon Aug 29 20:17:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:17:58.980635 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8433ad-35c7-4c40-adc3-e21d2b7237fe None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:58.981444 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8433ad-35c7-4c40-adc3-e21d2b7237fe None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:58.981583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8433ad-35c7-4c40-adc3-e21d2b7237fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:59.026879 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-ca8433ad-35c7-4c40-adc3-e21d2b7237fe None None] MFA Rules not processed for user `e0c76478b6694782b896cdf9cda32d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:17:59.027988 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ca8433ad-35c7-4c40-adc3-e21d2b7237fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:59.044972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1244/2486] 10.222.0.73 () {58 vars in 967 bytes} [Mon Aug 29 20:17:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:17:59.075259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b70782-32e2-4a47-9f01-9d0ee788718d service swift] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:59.135106 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b70782-32e2-4a47-9f01-9d0ee788718d service swift] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:59.146044 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfc155-14b2-4f0e-b74e-f4685474ba8b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92e30da8f394f3abc19b9e86ef126e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:59.146801 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfc155-14b2-4f0e-b74e-f4685474ba8b service swift] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:59.147253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfc155-14b2-4f0e-b74e-f4685474ba8b service swift] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:59.147641 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfc155-14b2-4f0e-b74e-f4685474ba8b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:59.169260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfc155-14b2-4f0e-b74e-f4685474ba8b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:59.174732 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfc155-14b2-4f0e-b74e-f4685474ba8b service swift] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:59.176949 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ac709-8c50-4f6d-841d-f605ffd147a0 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:59.177455 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ac709-8c50-4f6d-841d-f605ffd147a0 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:59.177995 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ac709-8c50-4f6d-841d-f605ffd147a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:59.190580 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1243/2487] 10.222.0.73 () {60 vars in 1384 bytes} [Mon Aug 29 20:17:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:17:59.244125 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:17:59.261765 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-3c7ac709-8c50-4f6d-841d-f605ffd147a0 None None] MFA Rules not processed for user `e0c76478b6694782b896cdf9cda32d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:17:59.263479 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-3c7ac709-8c50-4f6d-841d-f605ffd147a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:17:59.278516 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1245/2488] 10.222.0.73 () {58 vars in 967 bytes} [Mon Aug 29 20:17:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:17:59.306234 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfc155-14b2-4f0e-b74e-f4685474ba8b service swift] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:17:59.326731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfc155-14b2-4f0e-b74e-f4685474ba8b service swift] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:17:59.345403 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147e56d-b879-42d8-93d0-9f99e114c506 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92e30da8f394f3abc19b9e86ef126e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:17:59.346695 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147e56d-b879-42d8-93d0-9f99e114c506 service swift] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:17:59.349945 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147e56d-b879-42d8-93d0-9f99e114c506 service swift] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:17:59.350377 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147e56d-b879-42d8-93d0-9f99e114c506 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:17:59.371428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147e56d-b879-42d8-93d0-9f99e114c506 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:17:59.376708 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147e56d-b879-42d8-93d0-9f99e114c506 service swift] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:17:59.398349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1244/2489] 10.222.0.73 () {60 vars in 1384 bytes} [Mon Aug 29 20:17:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:18:02.629201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ac709-8c50-4f6d-841d-f605ffd147a0 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:02.631238 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-3c7ac709-8c50-4f6d-841d-f605ffd147a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:02.838968 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ac709-8c50-4f6d-841d-f605ffd147a0 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:02.845915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73882e04-7b0b-4fbb-a85d-e02a62b0b3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:02.847191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73882e04-7b0b-4fbb-a85d-e02a62b0b3d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:02.847818 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73882e04-7b0b-4fbb-a85d-e02a62b0b3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:02.848174 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73882e04-7b0b-4fbb-a85d-e02a62b0b3d8 admin admin] PATH_INFO: `/v3/users/b41427ee7f3a4b478f5587b41a3b9e5e` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:02.849673 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73882e04-7b0b-4fbb-a85d-e02a62b0b3d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b41427ee7f3a4b478f5587b41a3b9e5e)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:02.851921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73882e04-7b0b-4fbb-a85d-e02a62b0b3d8 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:02.961066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-73882e04-7b0b-4fbb-a85d-e02a62b0b3d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b41427ee7f3a4b478f5587b41a3b9e5e'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:18:03.000131 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-73882e04-7b0b-4fbb-a85d-e02a62b0b3d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b41427ee7f3a4b478f5587b41a3b9e5e'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:18:03.016386 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-73882e04-7b0b-4fbb-a85d-e02a62b0b3d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b41427ee7f3a4b478f5587b41a3b9e5e'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:18:03.040637 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-73882e04-7b0b-4fbb-a85d-e02a62b0b3d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b41427ee7f3a4b478f5587b41a3b9e5e'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:18:03.044416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-73882e04-7b0b-4fbb-a85d-e02a62b0b3d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b41427ee7f3a4b478f5587b41a3b9e5e'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:18:03.067419 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1246/2490] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:18:02 2022] DELETE /identity/v3/users/b41427ee7f3a4b478f5587b41a3b9e5e => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:18:03.074507 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7bccb-6e30-4652-adbb-8d709300d9a2 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:03.075056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7bccb-6e30-4652-adbb-8d709300d9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:03.075453 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7bccb-6e30-4652-adbb-8d709300d9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:03.131144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-ffb7bccb-6e30-4652-adbb-8d709300d9a2 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:03.134041 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-ffb7bccb-6e30-4652-adbb-8d709300d9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:03.317935 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1245/2491] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:18:03.331977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73882e04-7b0b-4fbb-a85d-e02a62b0b3d8 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:03.333961 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-73882e04-7b0b-4fbb-a85d-e02a62b0b3d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:03.543290 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73882e04-7b0b-4fbb-a85d-e02a62b0b3d8 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:03.550882 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f5f3e-ed31-42cf-b57a-30ab4bbb1169 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:03.552187 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f5f3e-ed31-42cf-b57a-30ab4bbb1169 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:03.552599 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f5f3e-ed31-42cf-b57a-30ab4bbb1169 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:03.553023 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f5f3e-ed31-42cf-b57a-30ab4bbb1169 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:03.568806 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f5f3e-ed31-42cf-b57a-30ab4bbb1169 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:03.570914 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f5f3e-ed31-42cf-b57a-30ab4bbb1169 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:03.584543 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1247/2492] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:18:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:18:04.098661 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7bccb-6e30-4652-adbb-8d709300d9a2 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:04.103350 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-ffb7bccb-6e30-4652-adbb-8d709300d9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:04.131687 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7bccb-6e30-4652-adbb-8d709300d9a2 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:04.158417 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7273602-a329-4184-892a-cd482a08fb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:04.164401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7273602-a329-4184-892a-cd482a08fb63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:04.164930 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7273602-a329-4184-892a-cd482a08fb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:04.165426 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7273602-a329-4184-892a-cd482a08fb63 admin admin] PATH_INFO: `/v3/projects/1b6e748b13ea4fc5ad88eaee50ffa437` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:04.167031 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7273602-a329-4184-892a-cd482a08fb63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b6e748b13ea4fc5ad88eaee50ffa437)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:04.170084 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7273602-a329-4184-892a-cd482a08fb63 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:04.265341 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b7273602-a329-4184-892a-cd482a08fb63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b6e748b13ea4fc5ad88eaee50ffa437', 'request_id': 'req-b7273602-a329-4184-892a-cd482a08fb63'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:18:04.277599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b7273602-a329-4184-892a-cd482a08fb63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b6e748b13ea4fc5ad88eaee50ffa437', 'request_id': 'req-b7273602-a329-4184-892a-cd482a08fb63'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:18:04.280598 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:04.304479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b7273602-a329-4184-892a-cd482a08fb63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b6e748b13ea4fc5ad88eaee50ffa437', 'request_id': 'req-b7273602-a329-4184-892a-cd482a08fb63'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:18:04.340468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b7273602-a329-4184-892a-cd482a08fb63 admin admin] The token cache is being invalidate because project 1b6e748b13ea4fc5ad88eaee50ffa437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:18:04.351257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b7273602-a329-4184-892a-cd482a08fb63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:18:04.352544 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b7273602-a329-4184-892a-cd482a08fb63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:18:04.356353 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1246/2493] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:18:04 2022] DELETE /identity/v3/projects/1b6e748b13ea4fc5ad88eaee50ffa437 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:18:05.834437 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ed8a1-7bad-4313-9560-6815d3e8d4e2 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:05.835253 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ed8a1-7bad-4313-9560-6815d3e8d4e2 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:05.835253 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ed8a1-7bad-4313-9560-6815d3e8d4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:05.877234 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-3f3ed8a1-7bad-4313-9560-6815d3e8d4e2 None None] MFA Rules not processed for user `a73812ad386243e29fc41ef97037ae85`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:05.878941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-3f3ed8a1-7bad-4313-9560-6815d3e8d4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:05.895710 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:06.010880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1248/2494] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:18:05 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:18:06.033861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7273602-a329-4184-892a-cd482a08fb63 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:06.040219 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-b7273602-a329-4184-892a-cd482a08fb63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:06.178994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7273602-a329-4184-892a-cd482a08fb63 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:06.188126 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3ba73-5b71-4700-85b6-e32b949296c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:06.189803 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3ba73-5b71-4700-85b6-e32b949296c6 service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:06.190558 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3ba73-5b71-4700-85b6-e32b949296c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:06.191056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3ba73-5b71-4700-85b6-e32b949296c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:06.193750 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-4fe3ba73-5b71-4700-85b6-e32b949296c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:06.283689 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3ba73-5b71-4700-85b6-e32b949296c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:06.286653 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3ba73-5b71-4700-85b6-e32b949296c6 service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:06.300245 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1247/2495] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:18:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:18:11.984907 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ed8a1-7bad-4313-9560-6815d3e8d4e2 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:11.987091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-3f3ed8a1-7bad-4313-9560-6815d3e8d4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:12.067362 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ed8a1-7bad-4313-9560-6815d3e8d4e2 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:12.071995 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f031124-d437-4d9b-9ccf-0a90ff4184a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:12.072927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f031124-d437-4d9b-9ccf-0a90ff4184a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:12.073255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f031124-d437-4d9b-9ccf-0a90ff4184a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:12.073656 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f031124-d437-4d9b-9ccf-0a90ff4184a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:12.076026 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-0f031124-d437-4d9b-9ccf-0a90ff4184a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:12.154457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f031124-d437-4d9b-9ccf-0a90ff4184a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:12.156259 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f031124-d437-4d9b-9ccf-0a90ff4184a8 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:12.173483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1249/2496] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:18:11 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:18:16.293969 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:DELETE=3,keystone:SELECT=60,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:21.243658 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3ba73-5b71-4700-85b6-e32b949296c6 service nova] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:21.246081 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-4fe3ba73-5b71-4700-85b6-e32b949296c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:21.279472 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3ba73-5b71-4700-85b6-e32b949296c6 service nova] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:21.289598 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cded73b-f9eb-4bef-86c0-2bf2359648f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:21.290788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cded73b-f9eb-4bef-86c0-2bf2359648f9 service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:21.291401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cded73b-f9eb-4bef-86c0-2bf2359648f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:21.291757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cded73b-f9eb-4bef-86c0-2bf2359648f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:21.293921 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9cded73b-f9eb-4bef-86c0-2bf2359648f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:21.309812 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cded73b-f9eb-4bef-86c0-2bf2359648f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:21.313232 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cded73b-f9eb-4bef-86c0-2bf2359648f9 service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:21.333197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1248/2497] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:18:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:18:22.167902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=72 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:26.914003 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6309d26-59b8-4614-a683-b8966a67acb1 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:26.914642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6309d26-59b8-4614-a683-b8966a67acb1 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:26.914642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6309d26-59b8-4614-a683-b8966a67acb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:26.938411 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-b6309d26-59b8-4614-a683-b8966a67acb1 None None] MFA Rules not processed for user `8640600ca0d745cda547a1027c947089`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:26.939992 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-b6309d26-59b8-4614-a683-b8966a67acb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:26.950061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1250/2498] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:18:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:18:27.325447 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cded73b-f9eb-4bef-86c0-2bf2359648f9 service nova] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:27.327023 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9cded73b-f9eb-4bef-86c0-2bf2359648f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:27.404438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cded73b-f9eb-4bef-86c0-2bf2359648f9 service nova] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:27.409857 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8d6d0-2e35-4016-a023-19e3f8290adf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:27.410514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8d6d0-2e35-4016-a023-19e3f8290adf service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:27.410918 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8d6d0-2e35-4016-a023-19e3f8290adf service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:27.411204 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8d6d0-2e35-4016-a023-19e3f8290adf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:27.414600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-4af8d6d0-2e35-4016-a023-19e3f8290adf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:27.426373 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8d6d0-2e35-4016-a023-19e3f8290adf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:27.427820 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8d6d0-2e35-4016-a023-19e3f8290adf service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:27.440435 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1249/2499] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:18:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:18:27.992429 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6309d26-59b8-4614-a683-b8966a67acb1 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:27.994007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-b6309d26-59b8-4614-a683-b8966a67acb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:28.087966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6309d26-59b8-4614-a683-b8966a67acb1 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:28.094219 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db913c-43d2-420f-a0da-871c895e1232 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab2e318b2854c46bc7ed2151a44a2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:28.094895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db913c-43d2-420f-a0da-871c895e1232 service placement] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:28.095176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db913c-43d2-420f-a0da-871c895e1232 service placement] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:28.096028 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db913c-43d2-420f-a0da-871c895e1232 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:28.098439 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-04db913c-43d2-420f-a0da-871c895e1232 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:28.112573 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db913c-43d2-420f-a0da-871c895e1232 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:28.114283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db913c-43d2-420f-a0da-871c895e1232 service placement] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:28.126342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1251/2500] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:18:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:18:31.132296 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8d6d0-2e35-4016-a023-19e3f8290adf service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:31.134115 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-4af8d6d0-2e35-4016-a023-19e3f8290adf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:31.156051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8d6d0-2e35-4016-a023-19e3f8290adf service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:31.162564 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d24f6-a516-47c6-9e45-b848f9ddc4d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:31.163413 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d24f6-a516-47c6-9e45-b848f9ddc4d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:31.163918 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d24f6-a516-47c6-9e45-b848f9ddc4d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:31.164685 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d24f6-a516-47c6-9e45-b848f9ddc4d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:31.167832 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-a94d24f6-a516-47c6-9e45-b848f9ddc4d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:31.183644 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d24f6-a516-47c6-9e45-b848f9ddc4d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:31.186848 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d24f6-a516-47c6-9e45-b848f9ddc4d1 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:31.200896 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1250/2501] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:18:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:18:35.659631 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db913c-43d2-420f-a0da-871c895e1232 service placement] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:35.680342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db913c-43d2-420f-a0da-871c895e1232 service placement] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:35.694696 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa08311-aa1c-4225-8f8b-d2b430ea61e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:35.695750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa08311-aa1c-4225-8f8b-d2b430ea61e8 service nova] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:35.696171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa08311-aa1c-4225-8f8b-d2b430ea61e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:35.696595 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa08311-aa1c-4225-8f8b-d2b430ea61e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:35.700514 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-0fa08311-aa1c-4225-8f8b-d2b430ea61e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:35.722734 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa08311-aa1c-4225-8f8b-d2b430ea61e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:35.725580 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa08311-aa1c-4225-8f8b-d2b430ea61e8 service nova] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:35.744593 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1252/2502] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:18:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:18:37.805527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d24f6-a516-47c6-9e45-b848f9ddc4d1 service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:37.817657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d24f6-a516-47c6-9e45-b848f9ddc4d1 service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:37.824639 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5e2b3-120d-4b16-99d4-3306cd8c662e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:37.826111 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5e2b3-120d-4b16-99d4-3306cd8c662e service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:37.826261 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5e2b3-120d-4b16-99d4-3306cd8c662e service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:37.826684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5e2b3-120d-4b16-99d4-3306cd8c662e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:37.829463 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9de5e2b3-120d-4b16-99d4-3306cd8c662e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:37.933512 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5e2b3-120d-4b16-99d4-3306cd8c662e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:37.935133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5e2b3-120d-4b16-99d4-3306cd8c662e service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:37.954326 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1251/2503] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:18:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:18:42.050493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa08311-aa1c-4225-8f8b-d2b430ea61e8 service nova] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:42.052243 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-0fa08311-aa1c-4225-8f8b-d2b430ea61e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:42.075605 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa08311-aa1c-4225-8f8b-d2b430ea61e8 service nova] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:42.081053 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0c45c-9db7-40df-ac09-db661a52478d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:42.082009 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0c45c-9db7-40df-ac09-db661a52478d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:42.082465 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0c45c-9db7-40df-ac09-db661a52478d admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:42.082738 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0c45c-9db7-40df-ac09-db661a52478d admin admin] PATH_INFO: `/v3/users/e75b537d44524de192282b10825293f9` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:42.084068 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0c45c-9db7-40df-ac09-db661a52478d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e75b537d44524de192282b10825293f9)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:42.086351 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0c45c-9db7-40df-ac09-db661a52478d admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:42.154878 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-7af0c45c-9db7-40df-ac09-db661a52478d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e75b537d44524de192282b10825293f9'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:18:42.175392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-7af0c45c-9db7-40df-ac09-db661a52478d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e75b537d44524de192282b10825293f9'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:18:42.186227 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-7af0c45c-9db7-40df-ac09-db661a52478d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e75b537d44524de192282b10825293f9'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:18:42.198906 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-7af0c45c-9db7-40df-ac09-db661a52478d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e75b537d44524de192282b10825293f9'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:18:42.199980 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-7af0c45c-9db7-40df-ac09-db661a52478d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e75b537d44524de192282b10825293f9'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:18:42.210484 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1253/2504] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:18:42 2022] DELETE /identity/v3/users/e75b537d44524de192282b10825293f9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:18:42.418661 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5e2b3-120d-4b16-99d4-3306cd8c662e service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:42.433054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5e2b3-120d-4b16-99d4-3306cd8c662e service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:42.512785 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e8d1f-1288-48a7-8cd2-5a832666a05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:42.513430 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e8d1f-1288-48a7-8cd2-5a832666a05a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:42.513717 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e8d1f-1288-48a7-8cd2-5a832666a05a admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:42.514013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e8d1f-1288-48a7-8cd2-5a832666a05a admin admin] PATH_INFO: `/v3/projects/bbb0263903e14938b44d9a6b430d84ce` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:42.515298 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e8d1f-1288-48a7-8cd2-5a832666a05a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbb0263903e14938b44d9a6b430d84ce)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:42.517433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e8d1f-1288-48a7-8cd2-5a832666a05a admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:42.530922 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:42.578737 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-107e8d1f-1288-48a7-8cd2-5a832666a05a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbb0263903e14938b44d9a6b430d84ce', 'request_id': 'req-107e8d1f-1288-48a7-8cd2-5a832666a05a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:18:42.588137 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-107e8d1f-1288-48a7-8cd2-5a832666a05a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbb0263903e14938b44d9a6b430d84ce', 'request_id': 'req-107e8d1f-1288-48a7-8cd2-5a832666a05a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:18:42.594323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-107e8d1f-1288-48a7-8cd2-5a832666a05a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbb0263903e14938b44d9a6b430d84ce', 'request_id': 'req-107e8d1f-1288-48a7-8cd2-5a832666a05a'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:18:42.608215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-107e8d1f-1288-48a7-8cd2-5a832666a05a admin admin] The token cache is being invalidate because project bbb0263903e14938b44d9a6b430d84ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:18:42.608592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-107e8d1f-1288-48a7-8cd2-5a832666a05a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:18:42.609306 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-107e8d1f-1288-48a7-8cd2-5a832666a05a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:18:42.611024 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1252/2505] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:18:42 2022] DELETE /identity/v3/projects/bbb0263903e14938b44d9a6b430d84ce => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:18:42.616891 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78620103-dd76-493f-bd16-ce9d3c0c8b3c None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:42.617175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78620103-dd76-493f-bd16-ce9d3c0c8b3c None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:42.617477 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78620103-dd76-493f-bd16-ce9d3c0c8b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:42.639281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-78620103-dd76-493f-bd16-ce9d3c0c8b3c None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:42.640502 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-78620103-dd76-493f-bd16-ce9d3c0c8b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:42.680453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:42.707091 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1254/2506] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:18:42.714965 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1b1cc-5065-4479-a0d9-9557c10254ac None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:42.715278 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1b1cc-5065-4479-a0d9-9557c10254ac None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:42.715557 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1b1cc-5065-4479-a0d9-9557c10254ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:42.736250 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-3ed1b1cc-5065-4479-a0d9-9557c10254ac None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:42.738172 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-3ed1b1cc-5065-4479-a0d9-9557c10254ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:42.746731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1253/2507] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:18:42.751040 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78620103-dd76-493f-bd16-ce9d3c0c8b3c None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:42.761359 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78620103-dd76-493f-bd16-ce9d3c0c8b3c None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:42.766126 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56136c-263b-4677-bb01-b13d59a32857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:42.766737 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56136c-263b-4677-bb01-b13d59a32857 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:42.767176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56136c-263b-4677-bb01-b13d59a32857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:42.767529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56136c-263b-4677-bb01-b13d59a32857 admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:42.768694 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56136c-263b-4677-bb01-b13d59a32857 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:18:42.768997 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56136c-263b-4677-bb01-b13d59a32857 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:42.770809 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56136c-263b-4677-bb01-b13d59a32857 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:42.784631 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1255/2508] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:18:42.789515 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1b1cc-5065-4479-a0d9-9557c10254ac None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:42.799628 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1b1cc-5065-4479-a0d9-9557c10254ac None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:42.804507 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e656f46d-1062-4809-952c-12a2128b99e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:42.805080 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656f46d-1062-4809-952c-12a2128b99e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:42.805436 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656f46d-1062-4809-952c-12a2128b99e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:42.806181 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656f46d-1062-4809-952c-12a2128b99e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:42.806699 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e656f46d-1062-4809-952c-12a2128b99e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:42.808660 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e656f46d-1062-4809-952c-12a2128b99e2 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:42.840449 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1254/2509] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:18:42 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:18:42.844755 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56136c-263b-4677-bb01-b13d59a32857 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:42.855159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56136c-263b-4677-bb01-b13d59a32857 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:42.920588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55261f-d45b-4d6d-b0ae-2e60f294ace3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:42.921194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55261f-d45b-4d6d-b0ae-2e60f294ace3 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:42.921464 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55261f-d45b-4d6d-b0ae-2e60f294ace3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:42.921755 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55261f-d45b-4d6d-b0ae-2e60f294ace3 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:42.922381 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d55261f-d45b-4d6d-b0ae-2e60f294ace3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:42.924272 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d55261f-d45b-4d6d-b0ae-2e60f294ace3 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:42.947205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1256/2510] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:18:42 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:18:42.951710 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e656f46d-1062-4809-952c-12a2128b99e2 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:42.963674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e656f46d-1062-4809-952c-12a2128b99e2 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:42.969934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac01ed-a489-4b81-9c09-cdd27402e57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:42.970544 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac01ed-a489-4b81-9c09-cdd27402e57c admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:42.970844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac01ed-a489-4b81-9c09-cdd27402e57c admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:42.971116 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac01ed-a489-4b81-9c09-cdd27402e57c admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:42.971992 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac01ed-a489-4b81-9c09-cdd27402e57c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:42.973572 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac01ed-a489-4b81-9c09-cdd27402e57c admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:42.991455 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1255/2511] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:18:42.992663 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55261f-d45b-4d6d-b0ae-2e60f294ace3 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:43.003750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55261f-d45b-4d6d-b0ae-2e60f294ace3 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:43.010559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6831b-c9e5-44c1-b96c-650568ca664b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:43.011705 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6831b-c9e5-44c1-b96c-650568ca664b admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:43.012286 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6831b-c9e5-44c1-b96c-650568ca664b admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:43.012790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6831b-c9e5-44c1-b96c-650568ca664b admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:43.014171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae6831b-c9e5-44c1-b96c-650568ca664b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:43.016983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae6831b-c9e5-44c1-b96c-650568ca664b admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:43.031915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1257/2512] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:18:43.036274 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac01ed-a489-4b81-9c09-cdd27402e57c admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:43.049147 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac01ed-a489-4b81-9c09-cdd27402e57c admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:43.054377 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47edb850-a220-42e9-a9a0-440d6f562dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:43.055036 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47edb850-a220-42e9-a9a0-440d6f562dca admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:43.055316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47edb850-a220-42e9-a9a0-440d6f562dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:43.055595 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47edb850-a220-42e9-a9a0-440d6f562dca admin admin] PATH_INFO: `/v3/projects/8ee0b0ec6aaf40ba8f5ba9e3abdeebb8/users/c7c8422c3d314adca25acf967ffd0998/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:43.072207 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47edb850-a220-42e9-a9a0-440d6f562dca admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ee0b0ec6aaf40ba8f5ba9e3abdeebb8, user_id=c7c8422c3d314adca25acf967ffd0998, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:43.075029 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47edb850-a220-42e9-a9a0-440d6f562dca admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:43.105059 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1256/2513] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:18:43 2022] PUT /identity/v3/projects/8ee0b0ec6aaf40ba8f5ba9e3abdeebb8/users/c7c8422c3d314adca25acf967ffd0998/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:18:43.107066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579280c-9712-4da1-9b7b-b76d7b2ba396 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:43.107597 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579280c-9712-4da1-9b7b-b76d7b2ba396 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:43.108012 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579280c-9712-4da1-9b7b-b76d7b2ba396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:43.133617 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-3579280c-9712-4da1-9b7b-b76d7b2ba396 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:43.135056 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-3579280c-9712-4da1-9b7b-b76d7b2ba396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:43.205033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1258/2514] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:18:43.215324 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47edb850-a220-42e9-a9a0-440d6f562dca admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:43.217482 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-47edb850-a220-42e9-a9a0-440d6f562dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:43.279293 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47edb850-a220-42e9-a9a0-440d6f562dca admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:43.283289 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d2b90-ce03-496c-a87a-b78b67a8a769 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:43.283921 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d2b90-ce03-496c-a87a-b78b67a8a769 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:43.284199 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d2b90-ce03-496c-a87a-b78b67a8a769 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:43.284591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d2b90-ce03-496c-a87a-b78b67a8a769 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:43.294210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d2b90-ce03-496c-a87a-b78b67a8a769 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:43.295736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d2b90-ce03-496c-a87a-b78b67a8a769 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:43.306514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1257/2515] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:18:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:18:47.295844 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579280c-9712-4da1-9b7b-b76d7b2ba396 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:47.297544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-3579280c-9712-4da1-9b7b-b76d7b2ba396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:47.400984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:47.453920 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579280c-9712-4da1-9b7b-b76d7b2ba396 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:47.485326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76876dc6-91cd-4ccb-bf9c-487253714049 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:47.487085 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76876dc6-91cd-4ccb-bf9c-487253714049 service nova] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:47.490628 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76876dc6-91cd-4ccb-bf9c-487253714049 service nova] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:47.491853 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76876dc6-91cd-4ccb-bf9c-487253714049 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:47.497314 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-76876dc6-91cd-4ccb-bf9c-487253714049 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:47.535659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76876dc6-91cd-4ccb-bf9c-487253714049 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:47.537330 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76876dc6-91cd-4ccb-bf9c-487253714049 service nova] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:47.557359 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1259/2516] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:18:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:18:48.606432 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038557a5-89d4-4625-964b-bc8b11a66c3a None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:48.607011 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038557a5-89d4-4625-964b-bc8b11a66c3a None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:48.607011 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038557a5-89d4-4625-964b-bc8b11a66c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:48.640973 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-038557a5-89d4-4625-964b-bc8b11a66c3a None None] MFA Rules not processed for user `c7c8422c3d314adca25acf967ffd0998`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:48.642276 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-038557a5-89d4-4625-964b-bc8b11a66c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:48.680208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:48.768621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1258/2517] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:18:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:18:48.796044 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76876dc6-91cd-4ccb-bf9c-487253714049 service nova] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:48.798471 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-76876dc6-91cd-4ccb-bf9c-487253714049 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:48.818897 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76876dc6-91cd-4ccb-bf9c-487253714049 service nova] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:48.826716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6489fd-858d-4fa4-8b6d-2d75cd5e7431 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:48.828035 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6489fd-858d-4fa4-8b6d-2d75cd5e7431 service nova] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:48.828310 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6489fd-858d-4fa4-8b6d-2d75cd5e7431 service nova] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:48.828742 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6489fd-858d-4fa4-8b6d-2d75cd5e7431 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:48.842124 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6489fd-858d-4fa4-8b6d-2d75cd5e7431 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:48.844262 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6489fd-858d-4fa4-8b6d-2d75cd5e7431 service nova] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:48.857750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1260/2518] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:18:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:18:50.972023 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94508d-1751-431d-9b0f-271f06287128 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:50.973195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94508d-1751-431d-9b0f-271f06287128 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:50.973631 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94508d-1751-431d-9b0f-271f06287128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:51.000472 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-cf94508d-1751-431d-9b0f-271f06287128 None None] MFA Rules not processed for user `c7c8422c3d314adca25acf967ffd0998`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:51.002353 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-cf94508d-1751-431d-9b0f-271f06287128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:51.010979 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1259/2519] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:18:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:18:51.436430 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6489fd-858d-4fa4-8b6d-2d75cd5e7431 service nova] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:51.463053 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6489fd-858d-4fa4-8b6d-2d75cd5e7431 service nova] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:51.470217 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cac046-9f2c-4c12-8cfb-962810afba35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:51.470952 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cac046-9f2c-4c12-8cfb-962810afba35 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:51.471289 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cac046-9f2c-4c12-8cfb-962810afba35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:51.471810 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cac046-9f2c-4c12-8cfb-962810afba35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:51.474522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-23cac046-9f2c-4c12-8cfb-962810afba35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:51.561119 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94508d-1751-431d-9b0f-271f06287128 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:51.568943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cac046-9f2c-4c12-8cfb-962810afba35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:51.570821 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cac046-9f2c-4c12-8cfb-962810afba35 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:51.572821 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94508d-1751-431d-9b0f-271f06287128 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:51.577631 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17178b14-d56e-43da-b272-1704011c933f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:51.578632 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17178b14-d56e-43da-b272-1704011c933f service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:51.578916 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17178b14-d56e-43da-b272-1704011c933f service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:51.579185 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17178b14-d56e-43da-b272-1704011c933f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:51.581256 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-17178b14-d56e-43da-b272-1704011c933f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:51.582369 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1261/2520] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:18:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:18:51.594301 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17178b14-d56e-43da-b272-1704011c933f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:51.601922 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17178b14-d56e-43da-b272-1704011c933f service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:51.613049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1260/2521] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:18:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:18:56.398897 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47fefe-5297-4ea5-aca1-28bc6f26c125 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:56.398897 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47fefe-5297-4ea5-aca1-28bc6f26c125 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:56.398897 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47fefe-5297-4ea5-aca1-28bc6f26c125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:56.422977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-9d47fefe-5297-4ea5-aca1-28bc6f26c125 None None] MFA Rules not processed for user `04c2aae2da284df29243b99d33725ddd`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:56.424221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-9d47fefe-5297-4ea5-aca1-28bc6f26c125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:56.491555 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1262/2522] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:18:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:19:01.607302 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=23 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:04.129401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdea8d1-8037-44f2-915d-8096d3a70b0f None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:04.129401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdea8d1-8037-44f2-915d-8096d3a70b0f None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:04.131931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdea8d1-8037-44f2-915d-8096d3a70b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:04.160064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-abdea8d1-8037-44f2-915d-8096d3a70b0f None None] MFA Rules not processed for user `1ffc617c50984f83adc3269b118c8a91`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:04.161336 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-abdea8d1-8037-44f2-915d-8096d3a70b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:04.238602 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1261/2523] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:19:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:19:06.483176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=73 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:14.229898 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=25 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:43.054767 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d19ea-54f4-441b-8617-601bfb166194 None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:43.055677 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d19ea-54f4-441b-8617-601bfb166194 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:43.055677 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d19ea-54f4-441b-8617-601bfb166194 None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:43.056893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1263/2524] 10.222.0.73 () {54 vars in 921 bytes} [Mon Aug 29 20:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:43.067905 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143f510-c32f-463a-92a7-5db9972116e4 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:43.068609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143f510-c32f-463a-92a7-5db9972116e4 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:43.068609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143f510-c32f-463a-92a7-5db9972116e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:43.090904 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-f143f510-c32f-463a-92a7-5db9972116e4 None None] MFA Rules not processed for user `64ba58d774674ed7b233ad6416db068a`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:43.092190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-f143f510-c32f-463a-92a7-5db9972116e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:43.105727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1262/2525] 10.222.0.73 () {58 vars in 1018 bytes} [Mon Aug 29 20:19:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:19:43.127296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d19ea-54f4-441b-8617-601bfb166194 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:43.130165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-650d19ea-54f4-441b-8617-601bfb166194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:43.145987 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d19ea-54f4-441b-8617-601bfb166194 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:43.150247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908405a-07f1-46c4-8403-69110c70a1d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:43.150865 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908405a-07f1-46c4-8403-69110c70a1d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:43.151188 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908405a-07f1-46c4-8403-69110c70a1d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:43.151547 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908405a-07f1-46c4-8403-69110c70a1d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:43.161578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908405a-07f1-46c4-8403-69110c70a1d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:43.162734 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908405a-07f1-46c4-8403-69110c70a1d3 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:43.172251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1264/2526] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:19:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:19:43.285802 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6254eb9-c6b8-4495-a38c-cb16200a41b1 None None] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:43.286394 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6254eb9-c6b8-4495-a38c-cb16200a41b1 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:43.286889 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6254eb9-c6b8-4495-a38c-cb16200a41b1 None None] PATH_INFO: `/` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:43.288126 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1263/2527] 10.222.0.73 () {54 vars in 921 bytes} [Mon Aug 29 20:19:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:43.294097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa7c47-9214-4cff-84e4-69977ec9765c None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:43.294462 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa7c47-9214-4cff-84e4-69977ec9765c None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:43.294655 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa7c47-9214-4cff-84e4-69977ec9765c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:43.316260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-f5fa7c47-9214-4cff-84e4-69977ec9765c None None] MFA Rules not processed for user `95cf77a92a174ff8b0503baf5c7e63be`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:43.317649 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f5fa7c47-9214-4cff-84e4-69977ec9765c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:43.327452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1265/2528] 10.222.0.73 () {58 vars in 1018 bytes} [Mon Aug 29 20:19:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:19:43.338735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6254eb9-c6b8-4495-a38c-cb16200a41b1 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:43.352065 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6254eb9-c6b8-4495-a38c-cb16200a41b1 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:43.356439 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4dfcb2-b2fe-410a-9929-3aec02d5658a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:43.357054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4dfcb2-b2fe-410a-9929-3aec02d5658a service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:43.357337 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4dfcb2-b2fe-410a-9929-3aec02d5658a service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:43.357607 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4dfcb2-b2fe-410a-9929-3aec02d5658a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:43.373270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4dfcb2-b2fe-410a-9929-3aec02d5658a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:43.375373 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4dfcb2-b2fe-410a-9929-3aec02d5658a service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:43.388686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1264/2529] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:19:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:19:53.311130 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=9 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:53.383260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=9 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:06.405164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa7c47-9214-4cff-84e4-69977ec9765c None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:06.418032 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa7c47-9214-4cff-84e4-69977ec9765c None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:06.425468 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51627ec3-bdee-4a38-88b0-f18dd2604756 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:06.426379 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51627ec3-bdee-4a38-88b0-f18dd2604756 service nova] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:06.426578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51627ec3-bdee-4a38-88b0-f18dd2604756 service nova] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:06.426885 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51627ec3-bdee-4a38-88b0-f18dd2604756 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:06.429051 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-51627ec3-bdee-4a38-88b0-f18dd2604756 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:06.442200 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51627ec3-bdee-4a38-88b0-f18dd2604756 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:06.443834 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51627ec3-bdee-4a38-88b0-f18dd2604756 service nova] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:06.454651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1266/2530] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:06 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:09.485258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98087096-1e81-477c-b1f5-e8c37fa47dbd None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:09.486210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98087096-1e81-477c-b1f5-e8c37fa47dbd None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:09.486210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98087096-1e81-477c-b1f5-e8c37fa47dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:09.518298 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-98087096-1e81-477c-b1f5-e8c37fa47dbd None None] MFA Rules not processed for user `c7c8422c3d314adca25acf967ffd0998`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:09.521073 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-98087096-1e81-477c-b1f5-e8c37fa47dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:09.533927 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1265/2531] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:20:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:16.449446 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=6 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:19.510605 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:28.898466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7cdb90-ae94-4065-9615-a8a50a5b1c96 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:28.899347 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7cdb90-ae94-4065-9615-a8a50a5b1c96 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:28.899539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7cdb90-ae94-4065-9615-a8a50a5b1c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:28.920512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-ba7cdb90-ae94-4065-9615-a8a50a5b1c96 None None] MFA Rules not processed for user `e0c76478b6694782b896cdf9cda32d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:28.921739 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ba7cdb90-ae94-4065-9615-a8a50a5b1c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:28.995667 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1267/2532] 10.222.0.73 () {58 vars in 1009 bytes} [Mon Aug 29 20:20:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:29.053971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98087096-1e81-477c-b1f5-e8c37fa47dbd None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:29.055576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-98087096-1e81-477c-b1f5-e8c37fa47dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:29.143706 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98087096-1e81-477c-b1f5-e8c37fa47dbd None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:29.161334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2058b2-cf93-4e34-bf88-4da6ea0f4dfc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92e30da8f394f3abc19b9e86ef126e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:29.161334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2058b2-cf93-4e34-bf88-4da6ea0f4dfc service swift] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:29.161334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2058b2-cf93-4e34-bf88-4da6ea0f4dfc service swift] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:29.161334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2058b2-cf93-4e34-bf88-4da6ea0f4dfc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:29.175302 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2058b2-cf93-4e34-bf88-4da6ea0f4dfc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:29.179699 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2058b2-cf93-4e34-bf88-4da6ea0f4dfc service swift] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:29.199993 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1266/2533] 10.222.0.73 () {60 vars in 1384 bytes} [Mon Aug 29 20:20:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:20:38.982274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=25 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:39.194794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=32 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:43.614755 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7cdb90-ae94-4065-9615-a8a50a5b1c96 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:43.647285 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7cdb90-ae94-4065-9615-a8a50a5b1c96 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:43.651199 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfbd3c-10f0-4490-ab24-7bf5179a2ec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:43.651817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfbd3c-10f0-4490-ab24-7bf5179a2ec1 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:43.652093 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfbd3c-10f0-4490-ab24-7bf5179a2ec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:43.652370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfbd3c-10f0-4490-ab24-7bf5179a2ec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:43.654251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-0cbfbd3c-10f0-4490-ab24-7bf5179a2ec1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:43.738011 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfbd3c-10f0-4490-ab24-7bf5179a2ec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:43.740169 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfbd3c-10f0-4490-ab24-7bf5179a2ec1 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:43.761152 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1268/2534] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:44.282561 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2058b2-cf93-4e34-bf88-4da6ea0f4dfc service swift] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:44.284271 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-2a2058b2-cf93-4e34-bf88-4da6ea0f4dfc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:44.514483 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2058b2-cf93-4e34-bf88-4da6ea0f4dfc service swift] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:44.520065 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cecab9e-cfdf-459d-a0ea-f2b6614ba3b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab2e318b2854c46bc7ed2151a44a2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:44.521536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecab9e-cfdf-459d-a0ea-f2b6614ba3b1 service placement] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:44.521959 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecab9e-cfdf-459d-a0ea-f2b6614ba3b1 service placement] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:44.522241 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecab9e-cfdf-459d-a0ea-f2b6614ba3b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:44.524708 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-3cecab9e-cfdf-459d-a0ea-f2b6614ba3b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:44.548542 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cecab9e-cfdf-459d-a0ea-f2b6614ba3b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:44.551647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cecab9e-cfdf-459d-a0ea-f2b6614ba3b1 service placement] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:44.577295 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1267/2535] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:47.175072 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cff4dc-2b68-4c9e-8110-30d1bf4a724f None None] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:47.175607 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cff4dc-2b68-4c9e-8110-30d1bf4a724f None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:47.176059 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cff4dc-2b68-4c9e-8110-30d1bf4a724f None None] PATH_INFO: `/` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:47.179371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1269/2536] 10.222.0.73 () {54 vars in 921 bytes} [Mon Aug 29 20:20:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:47.185228 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276eb909-eea6-459a-819b-9928218713ca None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:47.185940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276eb909-eea6-459a-819b-9928218713ca None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:47.185940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276eb909-eea6-459a-819b-9928218713ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:47.302999 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-276eb909-eea6-459a-819b-9928218713ca None None] MFA Rules not processed for user `95cf77a92a174ff8b0503baf5c7e63be`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:47.304612 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-276eb909-eea6-459a-819b-9928218713ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:47.352029 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1268/2537] 10.222.0.73 () {58 vars in 1018 bytes} [Mon Aug 29 20:20:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:47.373133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cff4dc-2b68-4c9e-8110-30d1bf4a724f None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:47.378762 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-19cff4dc-2b68-4c9e-8110-30d1bf4a724f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:47.550007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cff4dc-2b68-4c9e-8110-30d1bf4a724f None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:47.558048 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e4f66-2f26-4ada-abf3-30af58c93b28 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b64b4b172467692d02851b2955b81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:47.559178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e4f66-2f26-4ada-abf3-30af58c93b28 service glance] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:47.559669 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e4f66-2f26-4ada-abf3-30af58c93b28 service glance] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:47.560158 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e4f66-2f26-4ada-abf3-30af58c93b28 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:47.579631 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e4f66-2f26-4ada-abf3-30af58c93b28 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:47.583655 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e4f66-2f26-4ada-abf3-30af58c93b28 service glance] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:47.623655 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1270/2538] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:48.130675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276eb909-eea6-459a-819b-9928218713ca None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:48.133071 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-276eb909-eea6-459a-819b-9928218713ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:48.158339 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276eb909-eea6-459a-819b-9928218713ca None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:48.176214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164810a-4b75-4998-ae7b-a7b689fc375d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab2e318b2854c46bc7ed2151a44a2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:48.178996 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164810a-4b75-4998-ae7b-a7b689fc375d service placement] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:48.179513 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164810a-4b75-4998-ae7b-a7b689fc375d service placement] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:48.179820 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164810a-4b75-4998-ae7b-a7b689fc375d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:48.181901 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-a164810a-4b75-4998-ae7b-a7b689fc375d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:48.201754 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164810a-4b75-4998-ae7b-a7b689fc375d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:48.203674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164810a-4b75-4998-ae7b-a7b689fc375d service placement] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:48.228498 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1269/2539] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:48.252628 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e4f66-2f26-4ada-abf3-30af58c93b28 service glance] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:48.255428 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-837e4f66-2f26-4ada-abf3-30af58c93b28 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:48.278487 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e4f66-2f26-4ada-abf3-30af58c93b28 service glance] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:48.283572 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30a64a-b290-4d9c-9b22-e25ba0718b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:48.284240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30a64a-b290-4d9c-9b22-e25ba0718b59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:48.284593 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30a64a-b290-4d9c-9b22-e25ba0718b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:48.284869 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30a64a-b290-4d9c-9b22-e25ba0718b59 admin admin] PATH_INFO: `/v3/users/04c2aae2da284df29243b99d33725ddd` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:48.286620 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d30a64a-b290-4d9c-9b22-e25ba0718b59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04c2aae2da284df29243b99d33725ddd)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:48.288631 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d30a64a-b290-4d9c-9b22-e25ba0718b59 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:48.362917 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-9d30a64a-b290-4d9c-9b22-e25ba0718b59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04c2aae2da284df29243b99d33725ddd'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:20:48.386949 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-9d30a64a-b290-4d9c-9b22-e25ba0718b59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04c2aae2da284df29243b99d33725ddd'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:20:48.398695 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-9d30a64a-b290-4d9c-9b22-e25ba0718b59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04c2aae2da284df29243b99d33725ddd'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:20:48.407142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:48.412897 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-9d30a64a-b290-4d9c-9b22-e25ba0718b59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04c2aae2da284df29243b99d33725ddd'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:20:48.414398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-9d30a64a-b290-4d9c-9b22-e25ba0718b59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04c2aae2da284df29243b99d33725ddd'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:20:48.428537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1271/2540] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:20:48 2022] DELETE /identity/v3/users/04c2aae2da284df29243b99d33725ddd => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:20:48.662156 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164810a-4b75-4998-ae7b-a7b689fc375d service placement] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:48.678645 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164810a-4b75-4998-ae7b-a7b689fc375d service placement] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:48.722840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:48.774051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7768723-d8af-4b3a-8c8c-f3176f1a6e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:48.775097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7768723-d8af-4b3a-8c8c-f3176f1a6e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:48.775543 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7768723-d8af-4b3a-8c8c-f3176f1a6e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:48.776015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7768723-d8af-4b3a-8c8c-f3176f1a6e08 admin admin] PATH_INFO: `/v3/projects/6fb06a5fd3a3482d991c542ccc993e88` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:48.777937 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7768723-d8af-4b3a-8c8c-f3176f1a6e08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fb06a5fd3a3482d991c542ccc993e88)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:48.781445 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7768723-d8af-4b3a-8c8c-f3176f1a6e08 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:48.860947 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a7768723-d8af-4b3a-8c8c-f3176f1a6e08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fb06a5fd3a3482d991c542ccc993e88', 'request_id': 'req-a7768723-d8af-4b3a-8c8c-f3176f1a6e08'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:20:48.870119 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a7768723-d8af-4b3a-8c8c-f3176f1a6e08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fb06a5fd3a3482d991c542ccc993e88', 'request_id': 'req-a7768723-d8af-4b3a-8c8c-f3176f1a6e08'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:20:48.876925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a7768723-d8af-4b3a-8c8c-f3176f1a6e08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fb06a5fd3a3482d991c542ccc993e88', 'request_id': 'req-a7768723-d8af-4b3a-8c8c-f3176f1a6e08'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:20:48.894836 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a7768723-d8af-4b3a-8c8c-f3176f1a6e08 admin admin] The token cache is being invalidate because project 6fb06a5fd3a3482d991c542ccc993e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:20:48.895135 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a7768723-d8af-4b3a-8c8c-f3176f1a6e08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:20:48.896509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a7768723-d8af-4b3a-8c8c-f3176f1a6e08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:20:48.898208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1270/2541] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:20:48 2022] DELETE /identity/v3/projects/6fb06a5fd3a3482d991c542ccc993e88 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:20:48.903571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02530775-f101-4864-a7dc-d73d69bfa195 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:48.903840 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02530775-f101-4864-a7dc-d73d69bfa195 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:48.904115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02530775-f101-4864-a7dc-d73d69bfa195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:48.970567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-02530775-f101-4864-a7dc-d73d69bfa195 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:48.971828 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-02530775-f101-4864-a7dc-d73d69bfa195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:49.048380 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1272/2542] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:49.056783 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b4d65-a53c-4d27-aef7-2e6497008a22 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:49.057110 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b4d65-a53c-4d27-aef7-2e6497008a22 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:49.057386 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b4d65-a53c-4d27-aef7-2e6497008a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:49.082417 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-240b4d65-a53c-4d27-aef7-2e6497008a22 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:49.084484 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-240b4d65-a53c-4d27-aef7-2e6497008a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:49.092981 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1271/2543] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:20:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:49.097922 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02530775-f101-4864-a7dc-d73d69bfa195 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:49.118978 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02530775-f101-4864-a7dc-d73d69bfa195 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:49.124101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ea393-6814-414b-bffa-38bbce23e096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:49.124716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ea393-6814-414b-bffa-38bbce23e096 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:49.124990 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ea393-6814-414b-bffa-38bbce23e096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:49.125254 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ea393-6814-414b-bffa-38bbce23e096 admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:49.125973 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ea393-6814-414b-bffa-38bbce23e096 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:49.126275 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ea393-6814-414b-bffa-38bbce23e096 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:49.127885 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ea393-6814-414b-bffa-38bbce23e096 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:49.143149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1273/2544] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:20:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:49.151936 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b4d65-a53c-4d27-aef7-2e6497008a22 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:49.162544 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b4d65-a53c-4d27-aef7-2e6497008a22 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:49.167274 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ce666-1638-49b1-8e0a-eb4122381b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:49.167882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ce666-1638-49b1-8e0a-eb4122381b7b admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:49.168158 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ce666-1638-49b1-8e0a-eb4122381b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:49.168424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ce666-1638-49b1-8e0a-eb4122381b7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:49.169058 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ce666-1638-49b1-8e0a-eb4122381b7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:49.171244 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ce666-1638-49b1-8e0a-eb4122381b7b admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:49.206043 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1272/2545] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:20:49 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:20:49.214817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ea393-6814-414b-bffa-38bbce23e096 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:49.226536 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ea393-6814-414b-bffa-38bbce23e096 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:49.303350 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22820024-6bc0-4095-adfc-1ff3a0c732ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:49.304467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22820024-6bc0-4095-adfc-1ff3a0c732ca admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:49.304467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22820024-6bc0-4095-adfc-1ff3a0c732ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:49.304813 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22820024-6bc0-4095-adfc-1ff3a0c732ca admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:49.305161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22820024-6bc0-4095-adfc-1ff3a0c732ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:49.307136 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22820024-6bc0-4095-adfc-1ff3a0c732ca admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:49.339226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1274/2546] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:20:49 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:20:49.343894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ce666-1638-49b1-8e0a-eb4122381b7b admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:49.354857 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ce666-1638-49b1-8e0a-eb4122381b7b admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:49.359781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98b8b6-9d45-4964-ae9e-c4df4a49a775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:49.360397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98b8b6-9d45-4964-ae9e-c4df4a49a775 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:49.360680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98b8b6-9d45-4964-ae9e-c4df4a49a775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:49.360953 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98b8b6-9d45-4964-ae9e-c4df4a49a775 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:49.361606 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98b8b6-9d45-4964-ae9e-c4df4a49a775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:49.363265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98b8b6-9d45-4964-ae9e-c4df4a49a775 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:49.384417 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1273/2547] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:20:49.395335 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22820024-6bc0-4095-adfc-1ff3a0c732ca admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:49.406439 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22820024-6bc0-4095-adfc-1ff3a0c732ca admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:49.412632 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817c4a6-0007-4ad8-8549-0656ddd76f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:49.413320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817c4a6-0007-4ad8-8549-0656ddd76f0d admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:49.413640 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817c4a6-0007-4ad8-8549-0656ddd76f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:49.413983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817c4a6-0007-4ad8-8549-0656ddd76f0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:49.414764 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817c4a6-0007-4ad8-8549-0656ddd76f0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:49.418375 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817c4a6-0007-4ad8-8549-0656ddd76f0d admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:49.436793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1275/2548] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:20:49.441225 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98b8b6-9d45-4964-ae9e-c4df4a49a775 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:49.453910 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98b8b6-9d45-4964-ae9e-c4df4a49a775 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:49.459070 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9c399-b577-45b3-94dc-ef70525f9144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:49.459745 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9c399-b577-45b3-94dc-ef70525f9144 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:49.460021 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9c399-b577-45b3-94dc-ef70525f9144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:49.460294 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9c399-b577-45b3-94dc-ef70525f9144 admin admin] PATH_INFO: `/v3/projects/09c84765f43545a08c5a45550a46629e/users/ac972bf1d94243098f5ef845b89ca22d/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:49.477269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9c399-b577-45b3-94dc-ef70525f9144 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09c84765f43545a08c5a45550a46629e, user_id=ac972bf1d94243098f5ef845b89ca22d, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:49.485990 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9c399-b577-45b3-94dc-ef70525f9144 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:49.501545 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1274/2549] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:20:49 2022] PUT /identity/v3/projects/09c84765f43545a08c5a45550a46629e/users/ac972bf1d94243098f5ef845b89ca22d/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:20:49.512596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631ad2c-e75b-4c38-9a7e-e14b3bcbadca None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:49.512877 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631ad2c-e75b-4c38-9a7e-e14b3bcbadca None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:49.513219 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631ad2c-e75b-4c38-9a7e-e14b3bcbadca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:49.552421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-7631ad2c-e75b-4c38-9a7e-e14b3bcbadca None None] MFA Rules not processed for user `ac972bf1d94243098f5ef845b89ca22d`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:49.553798 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-7631ad2c-e75b-4c38-9a7e-e14b3bcbadca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:49.588263 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:49.631908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1276/2550] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:20:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:49.643339 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9c399-b577-45b3-94dc-ef70525f9144 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:49.645649 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-1ac9c399-b577-45b3-94dc-ef70525f9144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:49.706798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:49.733989 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9c399-b577-45b3-94dc-ef70525f9144 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:49.740344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8481fe3-3a6a-454f-89d6-f32eb934eea6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:49.741058 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8481fe3-3a6a-454f-89d6-f32eb934eea6 service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:49.741346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8481fe3-3a6a-454f-89d6-f32eb934eea6 service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:49.741618 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8481fe3-3a6a-454f-89d6-f32eb934eea6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:49.750966 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8481fe3-3a6a-454f-89d6-f32eb934eea6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:49.752597 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8481fe3-3a6a-454f-89d6-f32eb934eea6 service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:49.762169 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1275/2551] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:50.409135 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce56bba-a18d-4212-8298-7709a8492f9c None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:50.409973 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce56bba-a18d-4212-8298-7709a8492f9c None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:50.409973 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce56bba-a18d-4212-8298-7709a8492f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:50.435683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-fce56bba-a18d-4212-8298-7709a8492f9c None None] MFA Rules not processed for user `e0c76478b6694782b896cdf9cda32d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:50.436928 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-fce56bba-a18d-4212-8298-7709a8492f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:50.498927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1277/2552] 10.222.0.73 () {58 vars in 1009 bytes} [Mon Aug 29 20:20:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:50.513049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8481fe3-3a6a-454f-89d6-f32eb934eea6 service nova] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:50.514628 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-f8481fe3-3a6a-454f-89d6-f32eb934eea6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:50.603586 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8481fe3-3a6a-454f-89d6-f32eb934eea6 service nova] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:50.609600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67ceec-9bfc-456a-804b-1c7ccd78c33e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92e30da8f394f3abc19b9e86ef126e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:50.610591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67ceec-9bfc-456a-804b-1c7ccd78c33e service swift] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:50.611105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67ceec-9bfc-456a-804b-1c7ccd78c33e service swift] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:50.611423 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67ceec-9bfc-456a-804b-1c7ccd78c33e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:50.621341 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67ceec-9bfc-456a-804b-1c7ccd78c33e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:50.622988 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67ceec-9bfc-456a-804b-1c7ccd78c33e service swift] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:50.634033 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1276/2553] 10.222.0.73 () {60 vars in 1384 bytes} [Mon Aug 29 20:20:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:20:58.597864 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce56bba-a18d-4212-8298-7709a8492f9c None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:58.751541 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce56bba-a18d-4212-8298-7709a8492f9c None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:58.756983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c3759-d574-401f-80d7-e8b6db4c17b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:58.758276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c3759-d574-401f-80d7-e8b6db4c17b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:58.758669 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c3759-d574-401f-80d7-e8b6db4c17b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:58.759045 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c3759-d574-401f-80d7-e8b6db4c17b5 admin admin] PATH_INFO: `/v3/users/ac972bf1d94243098f5ef845b89ca22d` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:58.770298 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c3759-d574-401f-80d7-e8b6db4c17b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac972bf1d94243098f5ef845b89ca22d)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:58.773343 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c3759-d574-401f-80d7-e8b6db4c17b5 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:58.877871 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-465c3759-d574-401f-80d7-e8b6db4c17b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac972bf1d94243098f5ef845b89ca22d'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:20:58.910078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-465c3759-d574-401f-80d7-e8b6db4c17b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac972bf1d94243098f5ef845b89ca22d'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:20:58.929361 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-465c3759-d574-401f-80d7-e8b6db4c17b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac972bf1d94243098f5ef845b89ca22d'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:20:58.929361 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:58.945842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-465c3759-d574-401f-80d7-e8b6db4c17b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac972bf1d94243098f5ef845b89ca22d'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:20:58.946771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-465c3759-d574-401f-80d7-e8b6db4c17b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac972bf1d94243098f5ef845b89ca22d'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:20:58.953692 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1278/2554] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:20:58 2022] DELETE /identity/v3/users/ac972bf1d94243098f5ef845b89ca22d => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:20:58.958579 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf49d6-853b-4e2c-b712-ad8c398dd1f8 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:58.959274 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf49d6-853b-4e2c-b712-ad8c398dd1f8 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:58.959274 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf49d6-853b-4e2c-b712-ad8c398dd1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:58.980801 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-6bbf49d6-853b-4e2c-b712-ad8c398dd1f8 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:58.982272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-6bbf49d6-853b-4e2c-b712-ad8c398dd1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:59.057393 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1277/2555] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:59.067265 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c3759-d574-401f-80d7-e8b6db4c17b5 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:59.069506 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-465c3759-d574-401f-80d7-e8b6db4c17b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:59.205766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c3759-d574-401f-80d7-e8b6db4c17b5 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:59.210399 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ce932-79ee-47ae-9da3-eb9742ab5af7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:59.211112 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ce932-79ee-47ae-9da3-eb9742ab5af7 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:59.211398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ce932-79ee-47ae-9da3-eb9742ab5af7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:59.211681 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ce932-79ee-47ae-9da3-eb9742ab5af7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:59.228773 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ce932-79ee-47ae-9da3-eb9742ab5af7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:59.230335 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ce932-79ee-47ae-9da3-eb9742ab5af7 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:59.242449 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1279/2556] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:59.662671 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf49d6-853b-4e2c-b712-ad8c398dd1f8 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:59.667941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-6bbf49d6-853b-4e2c-b712-ad8c398dd1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:59.689026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf49d6-853b-4e2c-b712-ad8c398dd1f8 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:59.696925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b27a3-a136-448b-8b34-0b63d2becb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:59.698180 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b27a3-a136-448b-8b34-0b63d2becb7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:59.699049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b27a3-a136-448b-8b34-0b63d2becb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:59.699600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b27a3-a136-448b-8b34-0b63d2becb7c admin admin] PATH_INFO: `/v3/projects/09c84765f43545a08c5a45550a46629e` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:59.701819 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b27a3-a136-448b-8b34-0b63d2becb7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=09c84765f43545a08c5a45550a46629e)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:59.705489 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b27a3-a136-448b-8b34-0b63d2becb7c admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:59.781228 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-570b27a3-a136-448b-8b34-0b63d2becb7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09c84765f43545a08c5a45550a46629e', 'request_id': 'req-570b27a3-a136-448b-8b34-0b63d2becb7c'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:20:59.790820 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-570b27a3-a136-448b-8b34-0b63d2becb7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09c84765f43545a08c5a45550a46629e', 'request_id': 'req-570b27a3-a136-448b-8b34-0b63d2becb7c'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:20:59.798533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-570b27a3-a136-448b-8b34-0b63d2becb7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09c84765f43545a08c5a45550a46629e', 'request_id': 'req-570b27a3-a136-448b-8b34-0b63d2becb7c'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:20:59.800362 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:59.814252 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-570b27a3-a136-448b-8b34-0b63d2becb7c admin admin] The token cache is being invalidate because project 09c84765f43545a08c5a45550a46629e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:20:59.815280 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-570b27a3-a136-448b-8b34-0b63d2becb7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:20:59.815280 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-570b27a3-a136-448b-8b34-0b63d2becb7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:20:59.816798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1278/2557] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:20:59 2022] DELETE /identity/v3/projects/09c84765f43545a08c5a45550a46629e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:20:59.823761 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a2174-e0ed-43ca-998c-a976e5e859f1 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:59.824371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a2174-e0ed-43ca-998c-a976e5e859f1 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:59.824525 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a2174-e0ed-43ca-998c-a976e5e859f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:59.851294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-714a2174-e0ed-43ca-998c-a976e5e859f1 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:59.852553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-714a2174-e0ed-43ca-998c-a976e5e859f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:59.945146 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1280/2558] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:20:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:59.955244 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ba2c6-e957-4928-b99f-a5b4f332e5fd None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:59.955724 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ba2c6-e957-4928-b99f-a5b4f332e5fd None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:59.956187 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ba2c6-e957-4928-b99f-a5b4f332e5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:00.009997 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-6a7ba2c6-e957-4928-b99f-a5b4f332e5fd None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:00.012232 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-6a7ba2c6-e957-4928-b99f-a5b4f332e5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:00.026407 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1279/2559] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:20:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:00.031023 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a2174-e0ed-43ca-998c-a976e5e859f1 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:00.052411 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a2174-e0ed-43ca-998c-a976e5e859f1 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:00.064779 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c733f1f3-1f65-4066-9ec0-9a7f253b6413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:00.066019 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733f1f3-1f65-4066-9ec0-9a7f253b6413 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:00.066552 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733f1f3-1f65-4066-9ec0-9a7f253b6413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:00.067065 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733f1f3-1f65-4066-9ec0-9a7f253b6413 admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:00.068873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c733f1f3-1f65-4066-9ec0-9a7f253b6413 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:00.069514 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c733f1f3-1f65-4066-9ec0-9a7f253b6413 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:00.072796 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c733f1f3-1f65-4066-9ec0-9a7f253b6413 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:00.098434 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1281/2560] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:21:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:21:00.106091 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ba2c6-e957-4928-b99f-a5b4f332e5fd None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:00.123653 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ba2c6-e957-4928-b99f-a5b4f332e5fd None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:00.131836 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59a5f2-8d36-4b01-a239-4aff6280e507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:00.132717 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59a5f2-8d36-4b01-a239-4aff6280e507 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:00.133265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59a5f2-8d36-4b01-a239-4aff6280e507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:00.133742 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59a5f2-8d36-4b01-a239-4aff6280e507 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:00.134852 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59a5f2-8d36-4b01-a239-4aff6280e507 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:00.138079 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59a5f2-8d36-4b01-a239-4aff6280e507 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:00.186749 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1280/2561] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:21:00 2022] POST /identity/v3/projects => generated 357 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:00.189813 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c733f1f3-1f65-4066-9ec0-9a7f253b6413 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:00.210204 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c733f1f3-1f65-4066-9ec0-9a7f253b6413 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:00.284246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93aada6-0745-4feb-887f-fc4218862238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:00.284548 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93aada6-0745-4feb-887f-fc4218862238 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:00.284703 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93aada6-0745-4feb-887f-fc4218862238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:00.284915 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93aada6-0745-4feb-887f-fc4218862238 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:00.285641 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93aada6-0745-4feb-887f-fc4218862238 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:00.287760 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93aada6-0745-4feb-887f-fc4218862238 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:00.297751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:00.312247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1282/2562] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:21:00 2022] POST /identity/v3/users => generated 279 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:00.318528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59a5f2-8d36-4b01-a239-4aff6280e507 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:00.330543 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59a5f2-8d36-4b01-a239-4aff6280e507 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:00.339575 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8c7af-d3a4-4cba-8e6c-7091b337d1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:00.340266 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8c7af-d3a4-4cba-8e6c-7091b337d1ee admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:00.340601 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8c7af-d3a4-4cba-8e6c-7091b337d1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:00.340918 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8c7af-d3a4-4cba-8e6c-7091b337d1ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:00.341640 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8c7af-d3a4-4cba-8e6c-7091b337d1ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:00.343505 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8c7af-d3a4-4cba-8e6c-7091b337d1ee admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:00.355517 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1281/2563] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:21:00.362487 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93aada6-0745-4feb-887f-fc4218862238 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:00.374396 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93aada6-0745-4feb-887f-fc4218862238 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:00.379771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a32db52-54b4-43cd-8b3e-0d0a47b5d3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:00.380946 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32db52-54b4-43cd-8b3e-0d0a47b5d3a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:00.381268 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32db52-54b4-43cd-8b3e-0d0a47b5d3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:00.381870 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32db52-54b4-43cd-8b3e-0d0a47b5d3a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:00.382577 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a32db52-54b4-43cd-8b3e-0d0a47b5d3a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:00.384316 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a32db52-54b4-43cd-8b3e-0d0a47b5d3a2 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:00.399398 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1283/2564] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:21:00.404402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8c7af-d3a4-4cba-8e6c-7091b337d1ee admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:00.415953 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8c7af-d3a4-4cba-8e6c-7091b337d1ee admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:00.421150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0aa200-00ae-41fa-aed6-d59e046c2009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:00.422018 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0aa200-00ae-41fa-aed6-d59e046c2009 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:00.422309 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0aa200-00ae-41fa-aed6-d59e046c2009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:00.422587 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0aa200-00ae-41fa-aed6-d59e046c2009 admin admin] PATH_INFO: `/v3/projects/609677a293d049fe943ea53191d46cc2/users/ffcef6155175495bb6e2ee0d02b69166/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:00.438409 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0aa200-00ae-41fa-aed6-d59e046c2009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=609677a293d049fe943ea53191d46cc2, user_id=ffcef6155175495bb6e2ee0d02b69166, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:00.441336 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0aa200-00ae-41fa-aed6-d59e046c2009 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:00.457547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1282/2565] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:21:00 2022] PUT /identity/v3/projects/609677a293d049fe943ea53191d46cc2/users/ffcef6155175495bb6e2ee0d02b69166/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:00.470235 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a13fe-35d8-449d-a65e-d8d5c36a7894 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:00.470520 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a13fe-35d8-449d-a65e-d8d5c36a7894 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:00.470856 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a13fe-35d8-449d-a65e-d8d5c36a7894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:00.502815 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-675a13fe-35d8-449d-a65e-d8d5c36a7894 None None] MFA Rules not processed for user `ffcef6155175495bb6e2ee0d02b69166`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:00.504374 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-675a13fe-35d8-449d-a65e-d8d5c36a7894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:00.581481 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1284/2566] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:00 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:00.587502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0aa200-00ae-41fa-aed6-d59e046c2009 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:00.661504 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0aa200-00ae-41fa-aed6-d59e046c2009 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:00.666787 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc844252-7c9c-428c-9bc4-809d9ccf4345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:00.667458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc844252-7c9c-428c-9bc4-809d9ccf4345 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:00.667792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc844252-7c9c-428c-9bc4-809d9ccf4345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:00.668117 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc844252-7c9c-428c-9bc4-809d9ccf4345 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:00.668802 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc844252-7c9c-428c-9bc4-809d9ccf4345 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:00.671366 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc844252-7c9c-428c-9bc4-809d9ccf4345 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:00.706448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1283/2567] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:21:00 2022] POST /identity/v3/projects => generated 357 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:00.711320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a13fe-35d8-449d-a65e-d8d5c36a7894 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:00.722500 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a13fe-35d8-449d-a65e-d8d5c36a7894 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:00.784544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ab567-7a0b-455b-b08c-26b8edf21fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:00.785163 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ab567-7a0b-455b-b08c-26b8edf21fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:00.785443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ab567-7a0b-455b-b08c-26b8edf21fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:00.785740 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ab567-7a0b-455b-b08c-26b8edf21fd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:00.787227 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ab567-7a0b-455b-b08c-26b8edf21fd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:00.789173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ab567-7a0b-455b-b08c-26b8edf21fd7 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:00.812885 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1285/2568] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:21:00 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:21:00.817281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc844252-7c9c-428c-9bc4-809d9ccf4345 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:00.836769 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc844252-7c9c-428c-9bc4-809d9ccf4345 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:00.842152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e54ebb-4c26-49f8-a414-e89d424f9c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:00.842795 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e54ebb-4c26-49f8-a414-e89d424f9c64 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:00.843073 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e54ebb-4c26-49f8-a414-e89d424f9c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:00.847320 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e54ebb-4c26-49f8-a414-e89d424f9c64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:00.848025 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e54ebb-4c26-49f8-a414-e89d424f9c64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:00.850350 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e54ebb-4c26-49f8-a414-e89d424f9c64 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:00.862431 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1284/2569] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:21:00.868029 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ab567-7a0b-455b-b08c-26b8edf21fd7 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:00.882453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ab567-7a0b-455b-b08c-26b8edf21fd7 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:00.886898 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c1471-3887-4f09-ad01-2ee92264e570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:00.887590 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c1471-3887-4f09-ad01-2ee92264e570 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:00.887871 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c1471-3887-4f09-ad01-2ee92264e570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:00.888144 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c1471-3887-4f09-ad01-2ee92264e570 admin admin] PATH_INFO: `/v3/projects/8b4631e41f2a44e19afac383bf258cb3/users/e0fe4a737d464da3a7ce6233072e1c47/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:00.906211 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648c1471-3887-4f09-ad01-2ee92264e570 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b4631e41f2a44e19afac383bf258cb3, user_id=e0fe4a737d464da3a7ce6233072e1c47, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:00.908880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648c1471-3887-4f09-ad01-2ee92264e570 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:00.927243 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1286/2570] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:21:00 2022] PUT /identity/v3/projects/8b4631e41f2a44e19afac383bf258cb3/users/e0fe4a737d464da3a7ce6233072e1c47/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:00.933347 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e54ebb-4c26-49f8-a414-e89d424f9c64 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:00.947198 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e54ebb-4c26-49f8-a414-e89d424f9c64 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:01.053844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494fc80d-1cee-4473-84da-c2952e2b8313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:01.054755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494fc80d-1cee-4473-84da-c2952e2b8313 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:01.055182 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494fc80d-1cee-4473-84da-c2952e2b8313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:01.055643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494fc80d-1cee-4473-84da-c2952e2b8313 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:01.056775 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494fc80d-1cee-4473-84da-c2952e2b8313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:01.059723 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494fc80d-1cee-4473-84da-c2952e2b8313 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:01.063689 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:01.080855 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1285/2571] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:21:01.084325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c1471-3887-4f09-ad01-2ee92264e570 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:01.111579 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c1471-3887-4f09-ad01-2ee92264e570 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:01.120114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2d468-bea3-404c-bbb4-4ef8c0a35663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:01.121069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2d468-bea3-404c-bbb4-4ef8c0a35663 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:01.121514 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2d468-bea3-404c-bbb4-4ef8c0a35663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:01.122035 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2d468-bea3-404c-bbb4-4ef8c0a35663 admin admin] PATH_INFO: `/v3/projects/8b4631e41f2a44e19afac383bf258cb3/users/e0fe4a737d464da3a7ce6233072e1c47/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:01.124831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2d468-bea3-404c-bbb4-4ef8c0a35663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b4631e41f2a44e19afac383bf258cb3, user_id=e0fe4a737d464da3a7ce6233072e1c47, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:01.128836 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2d468-bea3-404c-bbb4-4ef8c0a35663 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:01.145213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1287/2572] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:21:01 2022] PUT /identity/v3/projects/8b4631e41f2a44e19afac383bf258cb3/users/e0fe4a737d464da3a7ce6233072e1c47/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:01.156926 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a66b04-197f-47aa-8a4a-5d85ed1114c9 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:01.157294 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a66b04-197f-47aa-8a4a-5d85ed1114c9 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:01.157481 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a66b04-197f-47aa-8a4a-5d85ed1114c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:01.201965 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-16a66b04-197f-47aa-8a4a-5d85ed1114c9 None None] MFA Rules not processed for user `e0fe4a737d464da3a7ce6233072e1c47`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:01.204088 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-16a66b04-197f-47aa-8a4a-5d85ed1114c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:01.320895 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1286/2573] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:01 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:01.330962 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2d468-bea3-404c-bbb4-4ef8c0a35663 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:01.333984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-45a2d468-bea3-404c-bbb4-4ef8c0a35663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:01.367265 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:01.450582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2d468-bea3-404c-bbb4-4ef8c0a35663 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:01.454355 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cca4ac-b5ec-437f-8443-1a9a2b308588 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:01.454962 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cca4ac-b5ec-437f-8443-1a9a2b308588 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:01.455234 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cca4ac-b5ec-437f-8443-1a9a2b308588 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:01.455503 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cca4ac-b5ec-437f-8443-1a9a2b308588 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:01.467424 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cca4ac-b5ec-437f-8443-1a9a2b308588 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:01.468856 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cca4ac-b5ec-437f-8443-1a9a2b308588 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:01.479266 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1288/2574] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:01 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:01.575183 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a66b04-197f-47aa-8a4a-5d85ed1114c9 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:01.588473 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a66b04-197f-47aa-8a4a-5d85ed1114c9 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:01.592765 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0952af-a2e0-4eee-88e3-790e0e031d8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:01.593489 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0952af-a2e0-4eee-88e3-790e0e031d8e service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:01.593866 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0952af-a2e0-4eee-88e3-790e0e031d8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:01.594144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0952af-a2e0-4eee-88e3-790e0e031d8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:01.664798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0952af-a2e0-4eee-88e3-790e0e031d8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:01.666309 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0952af-a2e0-4eee-88e3-790e0e031d8e service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:01.676610 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1287/2575] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:01 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:07.230051 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95a3c4-a46f-4450-83f6-6083bd74c281 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:07.230941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95a3c4-a46f-4450-83f6-6083bd74c281 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:07.230941 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95a3c4-a46f-4450-83f6-6083bd74c281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:07.251554 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-af95a3c4-a46f-4450-83f6-6083bd74c281 None None] MFA Rules not processed for user `8640600ca0d745cda547a1027c947089`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:07.252748 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-af95a3c4-a46f-4450-83f6-6083bd74c281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:07.337331 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1289/2576] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:21:08.352953 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d6571-5fe6-4695-b7fa-8cbc1fba097c None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:08.353628 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d6571-5fe6-4695-b7fa-8cbc1fba097c None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:08.353628 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d6571-5fe6-4695-b7fa-8cbc1fba097c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:08.374496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-ae5d6571-5fe6-4695-b7fa-8cbc1fba097c None None] MFA Rules not processed for user `e0c76478b6694782b896cdf9cda32d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:08.375747 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-ae5d6571-5fe6-4695-b7fa-8cbc1fba097c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:08.444358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1288/2577] 10.222.0.73 () {58 vars in 967 bytes} [Mon Aug 29 20:21:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:08.459613 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95a3c4-a46f-4450-83f6-6083bd74c281 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:08.461180 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-af95a3c4-a46f-4450-83f6-6083bd74c281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:08.580076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95a3c4-a46f-4450-83f6-6083bd74c281 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:08.586205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081319ab-60d1-4d05-9ae1-5b50f101c2f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92e30da8f394f3abc19b9e86ef126e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:08.586807 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081319ab-60d1-4d05-9ae1-5b50f101c2f7 service swift] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:08.587086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081319ab-60d1-4d05-9ae1-5b50f101c2f7 service swift] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:08.587358 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081319ab-60d1-4d05-9ae1-5b50f101c2f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:08.596216 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081319ab-60d1-4d05-9ae1-5b50f101c2f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:08.597842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081319ab-60d1-4d05-9ae1-5b50f101c2f7 service swift] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:08.608310 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1290/2578] 10.222.0.73 () {60 vars in 1384 bytes} [Mon Aug 29 20:21:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 20:21:17.799763 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d6571-5fe6-4695-b7fa-8cbc1fba097c None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:17.971447 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:18.024189 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d6571-5fe6-4695-b7fa-8cbc1fba097c None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:18.030746 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbf829-b1af-4979-835d-01e276072aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:18.031439 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbf829-b1af-4979-835d-01e276072aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:18.031781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbf829-b1af-4979-835d-01e276072aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:18.032141 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbf829-b1af-4979-835d-01e276072aba admin admin] PATH_INFO: `/v3/users/ffcef6155175495bb6e2ee0d02b69166` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:18.033464 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbf829-b1af-4979-835d-01e276072aba admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffcef6155175495bb6e2ee0d02b69166)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:18.036375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbf829-b1af-4979-835d-01e276072aba admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:18.122005 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-0bbbf829-b1af-4979-835d-01e276072aba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffcef6155175495bb6e2ee0d02b69166'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:18.133404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-0bbbf829-b1af-4979-835d-01e276072aba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffcef6155175495bb6e2ee0d02b69166'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:18.151249 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-0bbbf829-b1af-4979-835d-01e276072aba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffcef6155175495bb6e2ee0d02b69166'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:18.159244 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-0bbbf829-b1af-4979-835d-01e276072aba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffcef6155175495bb6e2ee0d02b69166'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:18.160027 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-0bbbf829-b1af-4979-835d-01e276072aba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffcef6155175495bb6e2ee0d02b69166'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:18.171765 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1289/2579] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:21:17 2022] DELETE /identity/v3/users/ffcef6155175495bb6e2ee0d02b69166 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:18.179320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca45e6c-6e11-410e-850d-04be92377930 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:18.179320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca45e6c-6e11-410e-850d-04be92377930 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:18.179320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca45e6c-6e11-410e-850d-04be92377930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:18.200592 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-6ca45e6c-6e11-410e-850d-04be92377930 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:18.202095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-6ca45e6c-6e11-410e-850d-04be92377930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:18.257225 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=100 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:18.307742 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1291/2580] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:18.323867 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbf829-b1af-4979-835d-01e276072aba admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:18.328224 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-0bbbf829-b1af-4979-835d-01e276072aba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:18.412157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbf829-b1af-4979-835d-01e276072aba admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:18.418818 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca0c33-f4e1-41a8-8a70-e03b1704b66b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:18.419834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca0c33-f4e1-41a8-8a70-e03b1704b66b service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:18.420041 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca0c33-f4e1-41a8-8a70-e03b1704b66b service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:18.420222 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca0c33-f4e1-41a8-8a70-e03b1704b66b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:18.432886 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca0c33-f4e1-41a8-8a70-e03b1704b66b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:18.434838 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca0c33-f4e1-41a8-8a70-e03b1704b66b service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:18.445931 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1290/2581] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:18.810514 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca45e6c-6e11-410e-850d-04be92377930 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:18.812673 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-6ca45e6c-6e11-410e-850d-04be92377930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:18.829154 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca45e6c-6e11-410e-850d-04be92377930 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:18.837926 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73153a3-64f0-43b4-afbe-9564bddcdfb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:18.839015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73153a3-64f0-43b4-afbe-9564bddcdfb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:18.839517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73153a3-64f0-43b4-afbe-9564bddcdfb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:18.839986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73153a3-64f0-43b4-afbe-9564bddcdfb0 admin admin] PATH_INFO: `/v3/projects/609677a293d049fe943ea53191d46cc2` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:18.842035 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73153a3-64f0-43b4-afbe-9564bddcdfb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=609677a293d049fe943ea53191d46cc2)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:18.845879 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73153a3-64f0-43b4-afbe-9564bddcdfb0 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:18.904450 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d73153a3-64f0-43b4-afbe-9564bddcdfb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '609677a293d049fe943ea53191d46cc2', 'request_id': 'req-d73153a3-64f0-43b4-afbe-9564bddcdfb0'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:18.911259 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d73153a3-64f0-43b4-afbe-9564bddcdfb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '609677a293d049fe943ea53191d46cc2', 'request_id': 'req-d73153a3-64f0-43b4-afbe-9564bddcdfb0'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:18.920664 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d73153a3-64f0-43b4-afbe-9564bddcdfb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '609677a293d049fe943ea53191d46cc2', 'request_id': 'req-d73153a3-64f0-43b4-afbe-9564bddcdfb0'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:18.935117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d73153a3-64f0-43b4-afbe-9564bddcdfb0 admin admin] The token cache is being invalidate because project 609677a293d049fe943ea53191d46cc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:21:18.935568 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d73153a3-64f0-43b4-afbe-9564bddcdfb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:18.936274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d73153a3-64f0-43b4-afbe-9564bddcdfb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:18.937846 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1292/2582] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:21:18 2022] DELETE /identity/v3/projects/609677a293d049fe943ea53191d46cc2 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:18.943447 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca0c33-f4e1-41a8-8a70-e03b1704b66b service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:18.953910 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca0c33-f4e1-41a8-8a70-e03b1704b66b service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:19.013368 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:19.031500 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3766b56-0896-46b7-9fa3-a5245f65a125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:19.032225 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3766b56-0896-46b7-9fa3-a5245f65a125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:19.032527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3766b56-0896-46b7-9fa3-a5245f65a125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:19.032824 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3766b56-0896-46b7-9fa3-a5245f65a125 admin admin] PATH_INFO: `/v3/users/e0fe4a737d464da3a7ce6233072e1c47` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:19.034601 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3766b56-0896-46b7-9fa3-a5245f65a125 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0fe4a737d464da3a7ce6233072e1c47)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:19.036967 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3766b56-0896-46b7-9fa3-a5245f65a125 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:19.104162 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f3766b56-0896-46b7-9fa3-a5245f65a125 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0fe4a737d464da3a7ce6233072e1c47'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:19.115882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f3766b56-0896-46b7-9fa3-a5245f65a125 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0fe4a737d464da3a7ce6233072e1c47'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:19.132760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f3766b56-0896-46b7-9fa3-a5245f65a125 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0fe4a737d464da3a7ce6233072e1c47'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:19.142562 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f3766b56-0896-46b7-9fa3-a5245f65a125 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0fe4a737d464da3a7ce6233072e1c47'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:19.143494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f3766b56-0896-46b7-9fa3-a5245f65a125 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0fe4a737d464da3a7ce6233072e1c47'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:19.152022 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1291/2583] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:21:18 2022] DELETE /identity/v3/users/e0fe4a737d464da3a7ce6233072e1c47 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:19.512391 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73153a3-64f0-43b4-afbe-9564bddcdfb0 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:19.513973 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-d73153a3-64f0-43b4-afbe-9564bddcdfb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:19.531873 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73153a3-64f0-43b4-afbe-9564bddcdfb0 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:19.634885 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fde77-3dea-4e79-b364-160b6fcab3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:19.635808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fde77-3dea-4e79-b364-160b6fcab3d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:19.635991 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fde77-3dea-4e79-b364-160b6fcab3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:19.636269 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fde77-3dea-4e79-b364-160b6fcab3d5 admin admin] PATH_INFO: `/v3/projects/8b4631e41f2a44e19afac383bf258cb3` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:19.637673 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1fde77-3dea-4e79-b364-160b6fcab3d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b4631e41f2a44e19afac383bf258cb3)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:19.640052 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1fde77-3dea-4e79-b364-160b6fcab3d5 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:19.719886 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-7b1fde77-3dea-4e79-b364-160b6fcab3d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b4631e41f2a44e19afac383bf258cb3', 'request_id': 'req-7b1fde77-3dea-4e79-b364-160b6fcab3d5'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:19.734531 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-7b1fde77-3dea-4e79-b364-160b6fcab3d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b4631e41f2a44e19afac383bf258cb3', 'request_id': 'req-7b1fde77-3dea-4e79-b364-160b6fcab3d5'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:19.744000 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-7b1fde77-3dea-4e79-b364-160b6fcab3d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b4631e41f2a44e19afac383bf258cb3', 'request_id': 'req-7b1fde77-3dea-4e79-b364-160b6fcab3d5'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:19.745285 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:19.768775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-7b1fde77-3dea-4e79-b364-160b6fcab3d5 admin admin] The token cache is being invalidate because project 8b4631e41f2a44e19afac383bf258cb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:21:19.769534 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-7b1fde77-3dea-4e79-b364-160b6fcab3d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:19.770321 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-7b1fde77-3dea-4e79-b364-160b6fcab3d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:19.772861 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1293/2584] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:21:19 2022] DELETE /identity/v3/projects/8b4631e41f2a44e19afac383bf258cb3 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:19.781532 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c18d9-3d62-45f7-b9dc-438c04650a40 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:19.782015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c18d9-3d62-45f7-b9dc-438c04650a40 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:19.782605 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c18d9-3d62-45f7-b9dc-438c04650a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:19.810437 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-5b7c18d9-3d62-45f7-b9dc-438c04650a40 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:19.810884 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-5b7c18d9-3d62-45f7-b9dc-438c04650a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:19.897269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1292/2585] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:19.909426 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca974a3-c058-411e-9049-82f7d080fe6a None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:19.909789 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca974a3-c058-411e-9049-82f7d080fe6a None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:19.910130 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca974a3-c058-411e-9049-82f7d080fe6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:19.942874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-9ca974a3-c058-411e-9049-82f7d080fe6a None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:19.945224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-9ca974a3-c058-411e-9049-82f7d080fe6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:19.956619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1294/2586] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:19.961875 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c18d9-3d62-45f7-b9dc-438c04650a40 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:19.977753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c18d9-3d62-45f7-b9dc-438c04650a40 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:19.983642 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41105b-aa89-4296-b3c8-344fd96ba34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:19.984319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41105b-aa89-4296-b3c8-344fd96ba34f admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:19.984892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41105b-aa89-4296-b3c8-344fd96ba34f admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:19.985040 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41105b-aa89-4296-b3c8-344fd96ba34f admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:19.985891 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f41105b-aa89-4296-b3c8-344fd96ba34f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:19.986239 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f41105b-aa89-4296-b3c8-344fd96ba34f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:19.988280 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f41105b-aa89-4296-b3c8-344fd96ba34f admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:20.015491 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1293/2587] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:21:20.021505 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca974a3-c058-411e-9049-82f7d080fe6a None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:20.033108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca974a3-c058-411e-9049-82f7d080fe6a None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:20.039490 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df192dee-76da-4237-b3b5-af72b8106c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:20.040100 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df192dee-76da-4237-b3b5-af72b8106c90 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:20.040371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df192dee-76da-4237-b3b5-af72b8106c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:20.040636 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df192dee-76da-4237-b3b5-af72b8106c90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:20.041269 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df192dee-76da-4237-b3b5-af72b8106c90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:20.043205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df192dee-76da-4237-b3b5-af72b8106c90 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:20.079056 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1295/2588] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:21:20 2022] POST /identity/v3/projects => generated 387 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:20.084526 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41105b-aa89-4296-b3c8-344fd96ba34f admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:20.102054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41105b-aa89-4296-b3c8-344fd96ba34f admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:20.174152 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:20.185369 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac463a-b938-435c-b11d-dce3a174e56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:20.186478 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac463a-b938-435c-b11d-dce3a174e56c admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:20.187564 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac463a-b938-435c-b11d-dce3a174e56c admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:20.188335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac463a-b938-435c-b11d-dce3a174e56c admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:20.189502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac463a-b938-435c-b11d-dce3a174e56c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:20.194179 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac463a-b938-435c-b11d-dce3a174e56c admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:20.225386 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1294/2589] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:21:20 2022] POST /identity/v3/users => generated 294 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:20.229964 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df192dee-76da-4237-b3b5-af72b8106c90 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:20.243469 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df192dee-76da-4237-b3b5-af72b8106c90 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:20.249439 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23cfe6-9663-42c5-8140-0067a536f8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:20.250341 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23cfe6-9663-42c5-8140-0067a536f8c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:20.250795 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23cfe6-9663-42c5-8140-0067a536f8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:20.251179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23cfe6-9663-42c5-8140-0067a536f8c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:20.251974 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f23cfe6-9663-42c5-8140-0067a536f8c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:20.253976 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f23cfe6-9663-42c5-8140-0067a536f8c0 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:20.273400 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1296/2590] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:21:20.278660 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac463a-b938-435c-b11d-dce3a174e56c admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:20.291041 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac463a-b938-435c-b11d-dce3a174e56c admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:20.295870 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a3b4c-9f32-488a-91a2-e59c73d5476e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:20.296630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a3b4c-9f32-488a-91a2-e59c73d5476e admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:20.296961 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a3b4c-9f32-488a-91a2-e59c73d5476e admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:20.297451 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a3b4c-9f32-488a-91a2-e59c73d5476e admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:20.298108 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a3b4c-9f32-488a-91a2-e59c73d5476e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:20.299755 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a3b4c-9f32-488a-91a2-e59c73d5476e admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:20.310699 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1295/2591] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:21:20.314998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23cfe6-9663-42c5-8140-0067a536f8c0 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:20.329233 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23cfe6-9663-42c5-8140-0067a536f8c0 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:20.335243 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b92d3c-adc2-4831-944a-e3888f776573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:20.336277 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b92d3c-adc2-4831-944a-e3888f776573 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:20.336626 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b92d3c-adc2-4831-944a-e3888f776573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:20.337109 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b92d3c-adc2-4831-944a-e3888f776573 admin admin] PATH_INFO: `/v3/projects/b9cf22bef25341b5971ea86029942423/users/84f4b97877e642fb8c94353dc119569d/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:20.362213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b92d3c-adc2-4831-944a-e3888f776573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9cf22bef25341b5971ea86029942423, user_id=84f4b97877e642fb8c94353dc119569d, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:20.364999 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b92d3c-adc2-4831-944a-e3888f776573 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:20.382098 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1297/2592] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:21:20 2022] PUT /identity/v3/projects/b9cf22bef25341b5971ea86029942423/users/84f4b97877e642fb8c94353dc119569d/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:20.392567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce774c55-4782-435d-a999-48ca99817945 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:20.392878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce774c55-4782-435d-a999-48ca99817945 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:20.393185 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce774c55-4782-435d-a999-48ca99817945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:20.425860 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-ce774c55-4782-435d-a999-48ca99817945 None None] MFA Rules not processed for user `84f4b97877e642fb8c94353dc119569d`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:20.427033 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-ce774c55-4782-435d-a999-48ca99817945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:20.502925 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1296/2593] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:20.510583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b92d3c-adc2-4831-944a-e3888f776573 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:20.605085 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b92d3c-adc2-4831-944a-e3888f776573 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:20.610692 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81313b-e1ba-4ee6-b84d-e92a6ef308f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:20.611490 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81313b-e1ba-4ee6-b84d-e92a6ef308f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:20.611903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81313b-e1ba-4ee6-b84d-e92a6ef308f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:20.612326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81313b-e1ba-4ee6-b84d-e92a6ef308f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:20.613136 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81313b-e1ba-4ee6-b84d-e92a6ef308f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:20.615665 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81313b-e1ba-4ee6-b84d-e92a6ef308f1 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:20.655652 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1298/2594] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:21:20 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:20.667459 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce774c55-4782-435d-a999-48ca99817945 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:20.681299 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce774c55-4782-435d-a999-48ca99817945 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:20.764283 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4713f1-a21c-4cdd-ad34-865d9d5a7970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:20.765000 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4713f1-a21c-4cdd-ad34-865d9d5a7970 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:20.765344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4713f1-a21c-4cdd-ad34-865d9d5a7970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:20.765681 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4713f1-a21c-4cdd-ad34-865d9d5a7970 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:20.766440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4713f1-a21c-4cdd-ad34-865d9d5a7970 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:20.769205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4713f1-a21c-4cdd-ad34-865d9d5a7970 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:20.800361 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1297/2595] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:21:20 2022] POST /identity/v3/users => generated 294 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:21:20.802827 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81313b-e1ba-4ee6-b84d-e92a6ef308f1 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:20.815888 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81313b-e1ba-4ee6-b84d-e92a6ef308f1 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:20.823170 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28f8a1-2aaa-406d-8821-487790aa1a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:20.824047 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28f8a1-2aaa-406d-8821-487790aa1a0e admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:20.824974 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28f8a1-2aaa-406d-8821-487790aa1a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:20.825257 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28f8a1-2aaa-406d-8821-487790aa1a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:20.826014 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28f8a1-2aaa-406d-8821-487790aa1a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:20.827730 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28f8a1-2aaa-406d-8821-487790aa1a0e admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:20.845021 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1299/2596] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:21:20.852829 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4713f1-a21c-4cdd-ad34-865d9d5a7970 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:20.864249 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4713f1-a21c-4cdd-ad34-865d9d5a7970 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:20.872674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab477434-5a31-496e-899f-d6141b5d63d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:20.873439 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab477434-5a31-496e-899f-d6141b5d63d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:20.874001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab477434-5a31-496e-899f-d6141b5d63d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:20.874396 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab477434-5a31-496e-899f-d6141b5d63d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:20.875421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab477434-5a31-496e-899f-d6141b5d63d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:20.878269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab477434-5a31-496e-899f-d6141b5d63d3 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:20.891667 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1298/2597] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:21:20.897428 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28f8a1-2aaa-406d-8821-487790aa1a0e admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:20.912962 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28f8a1-2aaa-406d-8821-487790aa1a0e admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:20.918951 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9da89-ddfa-4af3-966d-f725633b4984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:20.919629 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9da89-ddfa-4af3-966d-f725633b4984 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:20.919910 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9da89-ddfa-4af3-966d-f725633b4984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:20.920190 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9da89-ddfa-4af3-966d-f725633b4984 admin admin] PATH_INFO: `/v3/projects/b352d18a533244f7bfcb44fe56ea8475/users/714370de2e2944a3adb2c7ecc858a6e2/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:20.946256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c9da89-ddfa-4af3-966d-f725633b4984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b352d18a533244f7bfcb44fe56ea8475, user_id=714370de2e2944a3adb2c7ecc858a6e2, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:20.951027 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c9da89-ddfa-4af3-966d-f725633b4984 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:20.978672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1300/2598] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:21:20 2022] PUT /identity/v3/projects/b352d18a533244f7bfcb44fe56ea8475/users/714370de2e2944a3adb2c7ecc858a6e2/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:20.989511 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7eb67-a347-45c1-b6eb-bd7f7a6c5350 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:20.990817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7eb67-a347-45c1-b6eb-bd7f7a6c5350 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:20.990817 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7eb67-a347-45c1-b6eb-bd7f7a6c5350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:21.030715 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-68b7eb67-a347-45c1-b6eb-bd7f7a6c5350 None None] MFA Rules not processed for user `714370de2e2944a3adb2c7ecc858a6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:21.032694 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-68b7eb67-a347-45c1-b6eb-bd7f7a6c5350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:21.086733 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:21.145792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1299/2599] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:21.151002 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9da89-ddfa-4af3-966d-f725633b4984 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:21.208799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:21.238369 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9da89-ddfa-4af3-966d-f725633b4984 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:21.244640 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60415d28-ff8c-4fe8-8e3f-6797d3d70941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:21.245262 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60415d28-ff8c-4fe8-8e3f-6797d3d70941 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:21.245546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60415d28-ff8c-4fe8-8e3f-6797d3d70941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:21.245859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60415d28-ff8c-4fe8-8e3f-6797d3d70941 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:21.246508 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60415d28-ff8c-4fe8-8e3f-6797d3d70941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:21.248443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60415d28-ff8c-4fe8-8e3f-6797d3d70941 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:21.280139 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1301/2600] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:21:21 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:21.284918 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7eb67-a347-45c1-b6eb-bd7f7a6c5350 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:21.299843 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7eb67-a347-45c1-b6eb-bd7f7a6c5350 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:21.364534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa2711-17a6-4384-bf2e-f399466b99d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:21.365757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa2711-17a6-4384-bf2e-f399466b99d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:21.366060 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa2711-17a6-4384-bf2e-f399466b99d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:21.366318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa2711-17a6-4384-bf2e-f399466b99d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:21.366967 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa2711-17a6-4384-bf2e-f399466b99d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:21.368883 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa2711-17a6-4384-bf2e-f399466b99d6 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:21.393792 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1300/2601] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:21:21 2022] POST /identity/v3/users => generated 300 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:21.399228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60415d28-ff8c-4fe8-8e3f-6797d3d70941 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:21.415156 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60415d28-ff8c-4fe8-8e3f-6797d3d70941 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:21.422459 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4e331-f48a-4798-bb36-65650be6e023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:21.423856 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4e331-f48a-4798-bb36-65650be6e023 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:21.424349 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4e331-f48a-4798-bb36-65650be6e023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:21.424794 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4e331-f48a-4798-bb36-65650be6e023 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:21.425904 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f4e331-f48a-4798-bb36-65650be6e023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:21.434654 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f4e331-f48a-4798-bb36-65650be6e023 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:21.449013 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1302/2602] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:21:21.454582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa2711-17a6-4384-bf2e-f399466b99d6 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:21.469173 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa2711-17a6-4384-bf2e-f399466b99d6 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:21.476062 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651ad3d-dc94-4194-b600-6006d7d287f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:21.476735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651ad3d-dc94-4194-b600-6006d7d287f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:21.477019 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651ad3d-dc94-4194-b600-6006d7d287f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:21.477328 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651ad3d-dc94-4194-b600-6006d7d287f8 admin admin] PATH_INFO: `/v3/projects/6acfe9b07f94479bb845949991e90cca/users/00079577d8cd40bb86e61c1dd603b89c/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:21.506176 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651ad3d-dc94-4194-b600-6006d7d287f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6acfe9b07f94479bb845949991e90cca, user_id=00079577d8cd40bb86e61c1dd603b89c, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:21.510367 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651ad3d-dc94-4194-b600-6006d7d287f8 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:21.529663 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1301/2603] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:21:21 2022] PUT /identity/v3/projects/6acfe9b07f94479bb845949991e90cca/users/00079577d8cd40bb86e61c1dd603b89c/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:21.534625 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4e331-f48a-4798-bb36-65650be6e023 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:21.546936 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4e331-f48a-4798-bb36-65650be6e023 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:21.630271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6baf68-5b0f-4608-93ed-3dc2f68ed0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:21.631209 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6baf68-5b0f-4608-93ed-3dc2f68ed0cf admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:21.631723 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6baf68-5b0f-4608-93ed-3dc2f68ed0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:21.632151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6baf68-5b0f-4608-93ed-3dc2f68ed0cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:21.632823 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6baf68-5b0f-4608-93ed-3dc2f68ed0cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:21.634869 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6baf68-5b0f-4608-93ed-3dc2f68ed0cf admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:21.654066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1303/2604] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:21:21.655465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651ad3d-dc94-4194-b600-6006d7d287f8 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:21.670071 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651ad3d-dc94-4194-b600-6006d7d287f8 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:21.677066 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c49db-4b63-4b64-9ff2-65f494eeea61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:21.678728 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c49db-4b63-4b64-9ff2-65f494eeea61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:21.679200 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c49db-4b63-4b64-9ff2-65f494eeea61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:21.679672 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c49db-4b63-4b64-9ff2-65f494eeea61 admin admin] PATH_INFO: `/v3/projects/6acfe9b07f94479bb845949991e90cca/users/00079577d8cd40bb86e61c1dd603b89c/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:21.683525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251c49db-4b63-4b64-9ff2-65f494eeea61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6acfe9b07f94479bb845949991e90cca, user_id=00079577d8cd40bb86e61c1dd603b89c, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:21.688939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251c49db-4b63-4b64-9ff2-65f494eeea61 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:21.710203 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1302/2605] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:21:21 2022] PUT /identity/v3/projects/6acfe9b07f94479bb845949991e90cca/users/00079577d8cd40bb86e61c1dd603b89c/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:21:21.722283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe1f44-4f70-493d-a679-177b3ddf2a5b None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:21.722606 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe1f44-4f70-493d-a679-177b3ddf2a5b None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:21.723089 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe1f44-4f70-493d-a679-177b3ddf2a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:21.764783 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-c6fe1f44-4f70-493d-a679-177b3ddf2a5b None None] MFA Rules not processed for user `00079577d8cd40bb86e61c1dd603b89c`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:21.766416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c6fe1f44-4f70-493d-a679-177b3ddf2a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:21.869070 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1304/2606] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:21 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:21:21.886197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c49db-4b63-4b64-9ff2-65f494eeea61 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:21.888661 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-251c49db-4b63-4b64-9ff2-65f494eeea61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:21.994235 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c49db-4b63-4b64-9ff2-65f494eeea61 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:21.998842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa4d9e-a965-47df-bddc-93981fc049a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:21.999641 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa4d9e-a965-47df-bddc-93981fc049a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:21.999986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa4d9e-a965-47df-bddc-93981fc049a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:22.000344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa4d9e-a965-47df-bddc-93981fc049a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:22.011498 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa4d9e-a965-47df-bddc-93981fc049a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:22.012942 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa4d9e-a965-47df-bddc-93981fc049a5 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:22.027596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1303/2607] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:21 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:22.841127 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe1f44-4f70-493d-a679-177b3ddf2a5b None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:22.858958 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe1f44-4f70-493d-a679-177b3ddf2a5b None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:22.868499 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc05ef-cf02-457a-99cf-2d3315a848b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:22.869180 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc05ef-cf02-457a-99cf-2d3315a848b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:22.869527 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc05ef-cf02-457a-99cf-2d3315a848b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:22.869907 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc05ef-cf02-457a-99cf-2d3315a848b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:22.916189 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:22.952195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc05ef-cf02-457a-99cf-2d3315a848b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:22.954264 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc05ef-cf02-457a-99cf-2d3315a848b2 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:22.972982 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1305/2608] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:22 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:21:27.533103 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa4d9e-a965-47df-bddc-93981fc049a5 service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:27.535533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c7fa4d9e-a965-47df-bddc-93981fc049a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:27.559323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa4d9e-a965-47df-bddc-93981fc049a5 service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:27.573097 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb06b8-9b56-4452-ae1f-4005e0c9253e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:27.574128 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb06b8-9b56-4452-ae1f-4005e0c9253e service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:27.574874 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb06b8-9b56-4452-ae1f-4005e0c9253e service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:27.575369 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb06b8-9b56-4452-ae1f-4005e0c9253e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:27.619078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:27.690208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb06b8-9b56-4452-ae1f-4005e0c9253e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:27.692592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb06b8-9b56-4452-ae1f-4005e0c9253e service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:27.728381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1304/2609] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:27 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:30.723721 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc05ef-cf02-457a-99cf-2d3315a848b2 service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:30.892000 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc05ef-cf02-457a-99cf-2d3315a848b2 service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:30.898215 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd28d2-6cea-4390-ba9d-a734437a4749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:30.898996 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd28d2-6cea-4390-ba9d-a734437a4749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:30.899277 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd28d2-6cea-4390-ba9d-a734437a4749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:30.899549 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd28d2-6cea-4390-ba9d-a734437a4749 admin admin] PATH_INFO: `/v3/users/84f4b97877e642fb8c94353dc119569d` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:30.901242 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd28d2-6cea-4390-ba9d-a734437a4749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84f4b97877e642fb8c94353dc119569d)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:30.903954 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd28d2-6cea-4390-ba9d-a734437a4749 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:30.995801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d9bd28d2-6cea-4390-ba9d-a734437a4749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84f4b97877e642fb8c94353dc119569d'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:31.014930 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d9bd28d2-6cea-4390-ba9d-a734437a4749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84f4b97877e642fb8c94353dc119569d'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:31.032443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d9bd28d2-6cea-4390-ba9d-a734437a4749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84f4b97877e642fb8c94353dc119569d'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:31.047377 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d9bd28d2-6cea-4390-ba9d-a734437a4749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84f4b97877e642fb8c94353dc119569d'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:31.048804 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d9bd28d2-6cea-4390-ba9d-a734437a4749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84f4b97877e642fb8c94353dc119569d'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:31.060067 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1306/2610] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:21:30 2022] DELETE /identity/v3/users/84f4b97877e642fb8c94353dc119569d => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:31.065422 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a59508-b637-4f87-b3bd-42423d3a579d None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:31.066260 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a59508-b637-4f87-b3bd-42423d3a579d None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:31.067024 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a59508-b637-4f87-b3bd-42423d3a579d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:31.106542 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-31a59508-b637-4f87-b3bd-42423d3a579d None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:31.108378 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-31a59508-b637-4f87-b3bd-42423d3a579d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:31.212954 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1305/2611] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:31.224129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd28d2-6cea-4390-ba9d-a734437a4749 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:31.226119 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-d9bd28d2-6cea-4390-ba9d-a734437a4749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:31.304715 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd28d2-6cea-4390-ba9d-a734437a4749 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:31.308469 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256495c-f090-48fa-bc02-d4a0c0f83afa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:31.309585 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256495c-f090-48fa-bc02-d4a0c0f83afa service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:31.310171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256495c-f090-48fa-bc02-d4a0c0f83afa service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:31.310552 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256495c-f090-48fa-bc02-d4a0c0f83afa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:31.318852 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:31.324352 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256495c-f090-48fa-bc02-d4a0c0f83afa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:31.326432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256495c-f090-48fa-bc02-d4a0c0f83afa service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:31.344282 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1307/2612] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:31.738658 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a59508-b637-4f87-b3bd-42423d3a579d None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:31.742041 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-31a59508-b637-4f87-b3bd-42423d3a579d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:31.771898 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a59508-b637-4f87-b3bd-42423d3a579d None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:31.777520 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2c5a6-18e5-4dbb-9d62-5f7381f2d1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:31.778220 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2c5a6-18e5-4dbb-9d62-5f7381f2d1f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:31.778496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2c5a6-18e5-4dbb-9d62-5f7381f2d1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:31.778764 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2c5a6-18e5-4dbb-9d62-5f7381f2d1f0 admin admin] PATH_INFO: `/v3/projects/b9cf22bef25341b5971ea86029942423` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:31.780111 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d2c5a6-18e5-4dbb-9d62-5f7381f2d1f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9cf22bef25341b5971ea86029942423)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:31.782109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d2c5a6-18e5-4dbb-9d62-5f7381f2d1f0 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:31.843206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-90d2c5a6-18e5-4dbb-9d62-5f7381f2d1f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9cf22bef25341b5971ea86029942423', 'request_id': 'req-90d2c5a6-18e5-4dbb-9d62-5f7381f2d1f0'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:31.850788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-90d2c5a6-18e5-4dbb-9d62-5f7381f2d1f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9cf22bef25341b5971ea86029942423', 'request_id': 'req-90d2c5a6-18e5-4dbb-9d62-5f7381f2d1f0'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:31.856877 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-90d2c5a6-18e5-4dbb-9d62-5f7381f2d1f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9cf22bef25341b5971ea86029942423', 'request_id': 'req-90d2c5a6-18e5-4dbb-9d62-5f7381f2d1f0'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:31.872539 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-90d2c5a6-18e5-4dbb-9d62-5f7381f2d1f0 admin admin] The token cache is being invalidate because project b9cf22bef25341b5971ea86029942423 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:21:31.872881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-90d2c5a6-18e5-4dbb-9d62-5f7381f2d1f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:31.873579 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-90d2c5a6-18e5-4dbb-9d62-5f7381f2d1f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:31.875180 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1306/2613] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:21:31 2022] DELETE /identity/v3/projects/b9cf22bef25341b5971ea86029942423 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:31.880383 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256495c-f090-48fa-bc02-d4a0c0f83afa service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:31.899132 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256495c-f090-48fa-bc02-d4a0c0f83afa service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:31.955942 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2c5a6-18e5-4dbb-9d62-5f7381f2d1f0 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:31.958141 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-90d2c5a6-18e5-4dbb-9d62-5f7381f2d1f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:32.018665 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e99f98-d3ea-46bb-85e9-69509c219542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:32.019325 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e99f98-d3ea-46bb-85e9-69509c219542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:32.019606 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e99f98-d3ea-46bb-85e9-69509c219542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:32.020182 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e99f98-d3ea-46bb-85e9-69509c219542 admin admin] PATH_INFO: `/v3/users/714370de2e2944a3adb2c7ecc858a6e2` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:32.023451 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e99f98-d3ea-46bb-85e9-69509c219542 admin admin] RBAC: Authorizing `identity:delete_user(user_id=714370de2e2944a3adb2c7ecc858a6e2)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:32.031210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e99f98-d3ea-46bb-85e9-69509c219542 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:32.106733 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2c5a6-18e5-4dbb-9d62-5f7381f2d1f0 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:32.111142 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:32.127011 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f96c46-94a4-4253-9f98-e5a9aa2abaa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:32.128725 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f96c46-94a4-4253-9f98-e5a9aa2abaa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:32.129441 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f96c46-94a4-4253-9f98-e5a9aa2abaa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:32.130100 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f96c46-94a4-4253-9f98-e5a9aa2abaa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:32.135627 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-13f96c46-94a4-4253-9f98-e5a9aa2abaa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:32.155857 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-16e99f98-d3ea-46bb-85e9-69509c219542 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '714370de2e2944a3adb2c7ecc858a6e2'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:32.181009 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-16e99f98-d3ea-46bb-85e9-69509c219542 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '714370de2e2944a3adb2c7ecc858a6e2'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:32.194409 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-16e99f98-d3ea-46bb-85e9-69509c219542 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '714370de2e2944a3adb2c7ecc858a6e2'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:32.206027 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-16e99f98-d3ea-46bb-85e9-69509c219542 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '714370de2e2944a3adb2c7ecc858a6e2'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:32.206831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-16e99f98-d3ea-46bb-85e9-69509c219542 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '714370de2e2944a3adb2c7ecc858a6e2'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:32.214715 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1308/2614] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:21:31 2022] DELETE /identity/v3/users/714370de2e2944a3adb2c7ecc858a6e2 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:32.245778 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f96c46-94a4-4253-9f98-e5a9aa2abaa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:32.247216 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f96c46-94a4-4253-9f98-e5a9aa2abaa6 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:32.266607 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1307/2615] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:21:32.781082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e99f98-d3ea-46bb-85e9-69509c219542 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:32.784298 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-16e99f98-d3ea-46bb-85e9-69509c219542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:32.882486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e99f98-d3ea-46bb-85e9-69509c219542 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:32.887735 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecefb0f2-f1cf-418a-927a-cc85377c5e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:32.888427 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefb0f2-f1cf-418a-927a-cc85377c5e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:32.889273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefb0f2-f1cf-418a-927a-cc85377c5e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:32.889273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefb0f2-f1cf-418a-927a-cc85377c5e85 admin admin] PATH_INFO: `/v3/projects/b352d18a533244f7bfcb44fe56ea8475` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:32.890370 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecefb0f2-f1cf-418a-927a-cc85377c5e85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b352d18a533244f7bfcb44fe56ea8475)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:32.892440 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecefb0f2-f1cf-418a-927a-cc85377c5e85 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:32.913462 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:32.972545 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-ecefb0f2-f1cf-418a-927a-cc85377c5e85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b352d18a533244f7bfcb44fe56ea8475', 'request_id': 'req-ecefb0f2-f1cf-418a-927a-cc85377c5e85'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:32.982085 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-ecefb0f2-f1cf-418a-927a-cc85377c5e85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b352d18a533244f7bfcb44fe56ea8475', 'request_id': 'req-ecefb0f2-f1cf-418a-927a-cc85377c5e85'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:32.996689 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-ecefb0f2-f1cf-418a-927a-cc85377c5e85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b352d18a533244f7bfcb44fe56ea8475', 'request_id': 'req-ecefb0f2-f1cf-418a-927a-cc85377c5e85'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:33.014684 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-ecefb0f2-f1cf-418a-927a-cc85377c5e85 admin admin] The token cache is being invalidate because project b352d18a533244f7bfcb44fe56ea8475 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:21:33.015114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-ecefb0f2-f1cf-418a-927a-cc85377c5e85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:33.015933 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-ecefb0f2-f1cf-418a-927a-cc85377c5e85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:33.018015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1309/2616] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:21:32 2022] DELETE /identity/v3/projects/b352d18a533244f7bfcb44fe56ea8475 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:33.023053 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f96c46-94a4-4253-9f98-e5a9aa2abaa6 service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:33.025895 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-13f96c46-94a4-4253-9f98-e5a9aa2abaa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:33.046123 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f96c46-94a4-4253-9f98-e5a9aa2abaa6 service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:33.128686 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49007b89-3e17-4a66-ada5-dc03cbbcb9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:33.129543 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49007b89-3e17-4a66-ada5-dc03cbbcb9d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:33.130336 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49007b89-3e17-4a66-ada5-dc03cbbcb9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:33.131575 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49007b89-3e17-4a66-ada5-dc03cbbcb9d6 admin admin] PATH_INFO: `/v3/users/00079577d8cd40bb86e61c1dd603b89c` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:33.134282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49007b89-3e17-4a66-ada5-dc03cbbcb9d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00079577d8cd40bb86e61c1dd603b89c)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:33.138269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49007b89-3e17-4a66-ada5-dc03cbbcb9d6 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:33.221182 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-49007b89-3e17-4a66-ada5-dc03cbbcb9d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00079577d8cd40bb86e61c1dd603b89c'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:33.232949 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-49007b89-3e17-4a66-ada5-dc03cbbcb9d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00079577d8cd40bb86e61c1dd603b89c'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:33.257059 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-49007b89-3e17-4a66-ada5-dc03cbbcb9d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00079577d8cd40bb86e61c1dd603b89c'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:33.272496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-49007b89-3e17-4a66-ada5-dc03cbbcb9d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00079577d8cd40bb86e61c1dd603b89c'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:33.273438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-49007b89-3e17-4a66-ada5-dc03cbbcb9d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00079577d8cd40bb86e61c1dd603b89c'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:33.281227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1308/2617] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:21:33 2022] DELETE /identity/v3/users/00079577d8cd40bb86e61c1dd603b89c => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:33.850095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecefb0f2-f1cf-418a-927a-cc85377c5e85 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:33.852039 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-ecefb0f2-f1cf-418a-927a-cc85377c5e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:33.898827 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecefb0f2-f1cf-418a-927a-cc85377c5e85 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:34.048384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236000f-b9cf-422a-83ee-d47dd93c63a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:34.049629 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236000f-b9cf-422a-83ee-d47dd93c63a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:34.049984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236000f-b9cf-422a-83ee-d47dd93c63a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:34.053323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236000f-b9cf-422a-83ee-d47dd93c63a9 admin admin] PATH_INFO: `/v3/projects/6acfe9b07f94479bb845949991e90cca` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:34.053323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1236000f-b9cf-422a-83ee-d47dd93c63a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6acfe9b07f94479bb845949991e90cca)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:34.057759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1236000f-b9cf-422a-83ee-d47dd93c63a9 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:34.170943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1236000f-b9cf-422a-83ee-d47dd93c63a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6acfe9b07f94479bb845949991e90cca', 'request_id': 'req-1236000f-b9cf-422a-83ee-d47dd93c63a9'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:34.181632 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1236000f-b9cf-422a-83ee-d47dd93c63a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6acfe9b07f94479bb845949991e90cca', 'request_id': 'req-1236000f-b9cf-422a-83ee-d47dd93c63a9'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:34.198191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1236000f-b9cf-422a-83ee-d47dd93c63a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6acfe9b07f94479bb845949991e90cca', 'request_id': 'req-1236000f-b9cf-422a-83ee-d47dd93c63a9'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:34.214152 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1236000f-b9cf-422a-83ee-d47dd93c63a9 admin admin] The token cache is being invalidate because project 6acfe9b07f94479bb845949991e90cca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:21:34.214462 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1236000f-b9cf-422a-83ee-d47dd93c63a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:34.215220 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1236000f-b9cf-422a-83ee-d47dd93c63a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:34.216738 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1310/2618] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:21:33 2022] DELETE /identity/v3/projects/6acfe9b07f94479bb845949991e90cca => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:34.223530 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ae9ba-80df-42c9-b070-c7e0725129cb None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:34.224364 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ae9ba-80df-42c9-b070-c7e0725129cb None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:34.224364 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ae9ba-80df-42c9-b070-c7e0725129cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:34.259310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-370ae9ba-80df-42c9-b070-c7e0725129cb None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:34.261054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-370ae9ba-80df-42c9-b070-c7e0725129cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:34.292496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:34.362307 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1309/2619] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:34.372670 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1625e-84ae-4ab4-95bc-ad8c3636ee07 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:34.372981 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1625e-84ae-4ab4-95bc-ad8c3636ee07 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:34.373966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1625e-84ae-4ab4-95bc-ad8c3636ee07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:34.398244 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-f0f1625e-84ae-4ab4-95bc-ad8c3636ee07 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:34.400018 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f0f1625e-84ae-4ab4-95bc-ad8c3636ee07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:34.408909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1311/2620] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:34.413645 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ae9ba-80df-42c9-b070-c7e0725129cb None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:34.429453 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ae9ba-80df-42c9-b070-c7e0725129cb None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:34.436002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2987ba1-e7cd-4165-af6c-9405e667cce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:34.436983 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2987ba1-e7cd-4165-af6c-9405e667cce4 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:34.437184 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2987ba1-e7cd-4165-af6c-9405e667cce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:34.437425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2987ba1-e7cd-4165-af6c-9405e667cce4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:34.438181 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2987ba1-e7cd-4165-af6c-9405e667cce4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:34.439024 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2987ba1-e7cd-4165-af6c-9405e667cce4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:34.440289 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2987ba1-e7cd-4165-af6c-9405e667cce4 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:34.461471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1310/2621] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:34.467993 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f1625e-84ae-4ab4-95bc-ad8c3636ee07 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:34.482164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f1625e-84ae-4ab4-95bc-ad8c3636ee07 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:34.497852 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41544c0e-b610-41f2-b495-e59d446441e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:34.498679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41544c0e-b610-41f2-b495-e59d446441e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:34.499048 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41544c0e-b610-41f2-b495-e59d446441e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:34.499498 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41544c0e-b610-41f2-b495-e59d446441e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:34.500276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41544c0e-b610-41f2-b495-e59d446441e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:34.502620 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41544c0e-b610-41f2-b495-e59d446441e9 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:34.546034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1312/2622] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:21:34 2022] POST /identity/v3/projects => generated 387 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:34.552081 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2987ba1-e7cd-4165-af6c-9405e667cce4 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:34.570466 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2987ba1-e7cd-4165-af6c-9405e667cce4 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:34.659566 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f098b-e36f-46de-bc9f-b905cb5f8168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:34.660848 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f098b-e36f-46de-bc9f-b905cb5f8168 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:34.661105 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f098b-e36f-46de-bc9f-b905cb5f8168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:34.661479 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f098b-e36f-46de-bc9f-b905cb5f8168 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:34.663079 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f098b-e36f-46de-bc9f-b905cb5f8168 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:34.665090 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f098b-e36f-46de-bc9f-b905cb5f8168 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:34.702475 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1311/2623] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:21:34 2022] POST /identity/v3/users => generated 294 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:34.708231 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41544c0e-b610-41f2-b495-e59d446441e9 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:34.723072 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41544c0e-b610-41f2-b495-e59d446441e9 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:34.730512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75bcac-b42c-441f-b90e-4530b2dd0f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:34.731542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75bcac-b42c-441f-b90e-4530b2dd0f8c admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:34.732077 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75bcac-b42c-441f-b90e-4530b2dd0f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:34.732539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75bcac-b42c-441f-b90e-4530b2dd0f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:34.733816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75bcac-b42c-441f-b90e-4530b2dd0f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:34.736235 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75bcac-b42c-441f-b90e-4530b2dd0f8c admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:34.737989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:34.757025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1313/2624] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:21:34.761811 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f098b-e36f-46de-bc9f-b905cb5f8168 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:34.777285 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f098b-e36f-46de-bc9f-b905cb5f8168 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:34.784670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e741b-d71e-444b-9cec-2ed8dfd538cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:34.785278 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e741b-d71e-444b-9cec-2ed8dfd538cd admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:34.785612 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e741b-d71e-444b-9cec-2ed8dfd538cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:34.786024 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e741b-d71e-444b-9cec-2ed8dfd538cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:34.786758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e741b-d71e-444b-9cec-2ed8dfd538cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:34.789164 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e741b-d71e-444b-9cec-2ed8dfd538cd admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:34.807343 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1312/2625] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:21:34.809076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75bcac-b42c-441f-b90e-4530b2dd0f8c admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:34.824626 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75bcac-b42c-441f-b90e-4530b2dd0f8c admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:34.834805 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473f7b5-b08f-4a87-b29e-6fbe8875709e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:34.835943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473f7b5-b08f-4a87-b29e-6fbe8875709e admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:34.836389 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473f7b5-b08f-4a87-b29e-6fbe8875709e admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:34.836760 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473f7b5-b08f-4a87-b29e-6fbe8875709e admin admin] PATH_INFO: `/v3/projects/02ee7259adde4d15828ae9b48120b276/users/a48475e1510d4b3da9e65e14a7dc051a/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:34.871861 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473f7b5-b08f-4a87-b29e-6fbe8875709e admin admin] RBAC: Authorizing `identity:create_grant(project_id=02ee7259adde4d15828ae9b48120b276, user_id=a48475e1510d4b3da9e65e14a7dc051a, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:34.874903 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473f7b5-b08f-4a87-b29e-6fbe8875709e admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:34.909793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1314/2626] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:21:34 2022] PUT /identity/v3/projects/02ee7259adde4d15828ae9b48120b276/users/a48475e1510d4b3da9e65e14a7dc051a/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:34.927709 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7554340-7b4d-45c2-b675-05fbbd130a7a None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:34.928716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7554340-7b4d-45c2-b675-05fbbd130a7a None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:34.929202 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7554340-7b4d-45c2-b675-05fbbd130a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:34.976320 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-d7554340-7b4d-45c2-b675-05fbbd130a7a None None] MFA Rules not processed for user `a48475e1510d4b3da9e65e14a7dc051a`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:34.977940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d7554340-7b4d-45c2-b675-05fbbd130a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:35.102452 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1313/2627] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:35.108828 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473f7b5-b08f-4a87-b29e-6fbe8875709e admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:35.202825 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473f7b5-b08f-4a87-b29e-6fbe8875709e admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:35.212927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e070cc3-d99b-4bfa-87d6-2626b0cbff07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:35.214009 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e070cc3-d99b-4bfa-87d6-2626b0cbff07 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:35.214415 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e070cc3-d99b-4bfa-87d6-2626b0cbff07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:35.214885 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e070cc3-d99b-4bfa-87d6-2626b0cbff07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:35.215948 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e070cc3-d99b-4bfa-87d6-2626b0cbff07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:35.219578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e070cc3-d99b-4bfa-87d6-2626b0cbff07 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:35.261281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1315/2628] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:21:35 2022] POST /identity/v3/projects => generated 387 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:35.267061 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7554340-7b4d-45c2-b675-05fbbd130a7a None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:35.284234 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7554340-7b4d-45c2-b675-05fbbd130a7a None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:35.304973 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:35.368502 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ac9e0-dc3a-4e7e-9b45-b08dc84098ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:35.369134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ac9e0-dc3a-4e7e-9b45-b08dc84098ed admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:35.369406 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ac9e0-dc3a-4e7e-9b45-b08dc84098ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:35.369676 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ac9e0-dc3a-4e7e-9b45-b08dc84098ed admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:35.370346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ac9e0-dc3a-4e7e-9b45-b08dc84098ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:35.372550 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ac9e0-dc3a-4e7e-9b45-b08dc84098ed admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:35.403814 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1314/2629] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:21:35 2022] POST /identity/v3/users => generated 300 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:35.405983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e070cc3-d99b-4bfa-87d6-2626b0cbff07 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:35.420453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e070cc3-d99b-4bfa-87d6-2626b0cbff07 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:35.425805 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60affa5-e6fa-4094-9f16-18dc15888051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:35.426503 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60affa5-e6fa-4094-9f16-18dc15888051 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:35.426839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60affa5-e6fa-4094-9f16-18dc15888051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:35.427151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60affa5-e6fa-4094-9f16-18dc15888051 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:35.427943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60affa5-e6fa-4094-9f16-18dc15888051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:35.430054 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60affa5-e6fa-4094-9f16-18dc15888051 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:35.449195 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1316/2630] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:21:35.450875 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ac9e0-dc3a-4e7e-9b45-b08dc84098ed admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:35.463861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ac9e0-dc3a-4e7e-9b45-b08dc84098ed admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:35.469346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd8c17-0f2b-4bff-b085-f562d3534abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:35.470056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd8c17-0f2b-4bff-b085-f562d3534abb admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:35.470334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd8c17-0f2b-4bff-b085-f562d3534abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:35.470608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd8c17-0f2b-4bff-b085-f562d3534abb admin admin] PATH_INFO: `/v3/projects/472d1ff4ac2a4e098cd7a16208f28ca0/users/0e737c9ad471462dbeef1b1230218d6a/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:35.492376 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fd8c17-0f2b-4bff-b085-f562d3534abb admin admin] RBAC: Authorizing `identity:create_grant(project_id=472d1ff4ac2a4e098cd7a16208f28ca0, user_id=0e737c9ad471462dbeef1b1230218d6a, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:35.495463 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fd8c17-0f2b-4bff-b085-f562d3534abb admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:35.515286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1315/2631] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:21:35 2022] PUT /identity/v3/projects/472d1ff4ac2a4e098cd7a16208f28ca0/users/0e737c9ad471462dbeef1b1230218d6a/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:35.520685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60affa5-e6fa-4094-9f16-18dc15888051 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:35.535199 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60affa5-e6fa-4094-9f16-18dc15888051 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:35.643362 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692cc35-409b-4c4e-be9c-ca569b2faa3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:35.644105 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692cc35-409b-4c4e-be9c-ca569b2faa3a admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:35.644519 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692cc35-409b-4c4e-be9c-ca569b2faa3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:35.644887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692cc35-409b-4c4e-be9c-ca569b2faa3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:35.645657 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692cc35-409b-4c4e-be9c-ca569b2faa3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:35.647848 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692cc35-409b-4c4e-be9c-ca569b2faa3a admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:35.661132 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1317/2632] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:21:35.666623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd8c17-0f2b-4bff-b085-f562d3534abb admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:35.678703 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd8c17-0f2b-4bff-b085-f562d3534abb admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:35.686456 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9466d3-128e-4024-af01-af746480cb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:35.687635 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9466d3-128e-4024-af01-af746480cb72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:35.688116 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9466d3-128e-4024-af01-af746480cb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:35.688561 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9466d3-128e-4024-af01-af746480cb72 admin admin] PATH_INFO: `/v3/projects/472d1ff4ac2a4e098cd7a16208f28ca0/users/0e737c9ad471462dbeef1b1230218d6a/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:35.691505 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9466d3-128e-4024-af01-af746480cb72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=472d1ff4ac2a4e098cd7a16208f28ca0, user_id=0e737c9ad471462dbeef1b1230218d6a, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:35.695232 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9466d3-128e-4024-af01-af746480cb72 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:35.709197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1316/2633] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:21:35 2022] PUT /identity/v3/projects/472d1ff4ac2a4e098cd7a16208f28ca0/users/0e737c9ad471462dbeef1b1230218d6a/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:35.726262 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a201e6-1af6-42fa-9d1e-399cbb34585d None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:35.726701 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a201e6-1af6-42fa-9d1e-399cbb34585d None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:35.727161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a201e6-1af6-42fa-9d1e-399cbb34585d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:35.778154 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-50a201e6-1af6-42fa-9d1e-399cbb34585d None None] MFA Rules not processed for user `0e737c9ad471462dbeef1b1230218d6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:35.780093 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-50a201e6-1af6-42fa-9d1e-399cbb34585d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:35.830842 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:35.889157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1318/2634] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:35 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:35.903767 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9466d3-128e-4024-af01-af746480cb72 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:35.905872 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-6a9466d3-128e-4024-af01-af746480cb72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:35.979910 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9466d3-128e-4024-af01-af746480cb72 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:35.984270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544bd430-cebb-467f-86b3-d573b0c61f58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:35.985436 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544bd430-cebb-467f-86b3-d573b0c61f58 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:35.985436 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544bd430-cebb-467f-86b3-d573b0c61f58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:35.985436 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544bd430-cebb-467f-86b3-d573b0c61f58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:35.995381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544bd430-cebb-467f-86b3-d573b0c61f58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:35.996788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544bd430-cebb-467f-86b3-d573b0c61f58 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:36.008273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1317/2635] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:35 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:36.179921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a201e6-1af6-42fa-9d1e-399cbb34585d None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:36.276631 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a201e6-1af6-42fa-9d1e-399cbb34585d None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:36.283922 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b53eb7-e357-474f-81f0-4f6e635a2624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:36.284767 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b53eb7-e357-474f-81f0-4f6e635a2624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:36.285029 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b53eb7-e357-474f-81f0-4f6e635a2624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:36.285301 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b53eb7-e357-474f-81f0-4f6e635a2624 admin admin] PATH_INFO: `/v3/users/a48475e1510d4b3da9e65e14a7dc051a` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:36.286758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b53eb7-e357-474f-81f0-4f6e635a2624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a48475e1510d4b3da9e65e14a7dc051a)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:36.289995 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b53eb7-e357-474f-81f0-4f6e635a2624 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:36.372866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-67b53eb7-e357-474f-81f0-4f6e635a2624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a48475e1510d4b3da9e65e14a7dc051a'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:36.386974 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-67b53eb7-e357-474f-81f0-4f6e635a2624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a48475e1510d4b3da9e65e14a7dc051a'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:36.397531 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-67b53eb7-e357-474f-81f0-4f6e635a2624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a48475e1510d4b3da9e65e14a7dc051a'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:36.412340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-67b53eb7-e357-474f-81f0-4f6e635a2624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a48475e1510d4b3da9e65e14a7dc051a'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:36.415062 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-67b53eb7-e357-474f-81f0-4f6e635a2624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a48475e1510d4b3da9e65e14a7dc051a'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:36.424393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1319/2636] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:21:36 2022] DELETE /identity/v3/users/a48475e1510d4b3da9e65e14a7dc051a => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:21:36.430382 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65904d3d-f3d8-4c75-a929-b6ed1b7d1217 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:36.431023 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65904d3d-f3d8-4c75-a929-b6ed1b7d1217 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:36.431023 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65904d3d-f3d8-4c75-a929-b6ed1b7d1217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:36.461303 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-65904d3d-f3d8-4c75-a929-b6ed1b7d1217 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:36.462832 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-65904d3d-f3d8-4c75-a929-b6ed1b7d1217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:36.552093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1318/2637] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:36.563775 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b53eb7-e357-474f-81f0-4f6e635a2624 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:36.566779 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-67b53eb7-e357-474f-81f0-4f6e635a2624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:36.663199 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:36.670754 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b53eb7-e357-474f-81f0-4f6e635a2624 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:36.676283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1cceb-a003-4e8b-b229-6b4808777e2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:36.677011 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1cceb-a003-4e8b-b229-6b4808777e2f service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:36.677660 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1cceb-a003-4e8b-b229-6b4808777e2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:36.678106 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1cceb-a003-4e8b-b229-6b4808777e2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:36.691666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1cceb-a003-4e8b-b229-6b4808777e2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:36.693485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1cceb-a003-4e8b-b229-6b4808777e2f service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:36.705375 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1320/2638] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:37.061151 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65904d3d-f3d8-4c75-a929-b6ed1b7d1217 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:37.063298 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-65904d3d-f3d8-4c75-a929-b6ed1b7d1217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:37.093189 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65904d3d-f3d8-4c75-a929-b6ed1b7d1217 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:37.100269 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ee044-206e-42e0-b6a9-1056b3ed58b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:37.101530 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ee044-206e-42e0-b6a9-1056b3ed58b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:37.102040 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ee044-206e-42e0-b6a9-1056b3ed58b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:37.102488 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ee044-206e-42e0-b6a9-1056b3ed58b8 admin admin] PATH_INFO: `/v3/projects/02ee7259adde4d15828ae9b48120b276` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:37.103803 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ee044-206e-42e0-b6a9-1056b3ed58b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02ee7259adde4d15828ae9b48120b276)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:37.105877 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ee044-206e-42e0-b6a9-1056b3ed58b8 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:37.117178 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:37.179198 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9f6ee044-206e-42e0-b6a9-1056b3ed58b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02ee7259adde4d15828ae9b48120b276', 'request_id': 'req-9f6ee044-206e-42e0-b6a9-1056b3ed58b8'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:37.186740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9f6ee044-206e-42e0-b6a9-1056b3ed58b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02ee7259adde4d15828ae9b48120b276', 'request_id': 'req-9f6ee044-206e-42e0-b6a9-1056b3ed58b8'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:37.192813 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9f6ee044-206e-42e0-b6a9-1056b3ed58b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02ee7259adde4d15828ae9b48120b276', 'request_id': 'req-9f6ee044-206e-42e0-b6a9-1056b3ed58b8'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:37.209824 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9f6ee044-206e-42e0-b6a9-1056b3ed58b8 admin admin] The token cache is being invalidate because project 02ee7259adde4d15828ae9b48120b276 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:21:37.210246 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9f6ee044-206e-42e0-b6a9-1056b3ed58b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:37.211307 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9f6ee044-206e-42e0-b6a9-1056b3ed58b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:37.213918 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1319/2639] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:21:37 2022] DELETE /identity/v3/projects/02ee7259adde4d15828ae9b48120b276 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:37.218475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1cceb-a003-4e8b-b229-6b4808777e2f service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:37.229678 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1cceb-a003-4e8b-b229-6b4808777e2f service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:37.309733 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c702b-1fe2-4dd7-855e-09fd4896b038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:37.311099 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c702b-1fe2-4dd7-855e-09fd4896b038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:37.311657 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c702b-1fe2-4dd7-855e-09fd4896b038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:37.312193 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c702b-1fe2-4dd7-855e-09fd4896b038 admin admin] PATH_INFO: `/v3/users/0e737c9ad471462dbeef1b1230218d6a` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:37.314386 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c702b-1fe2-4dd7-855e-09fd4896b038 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e737c9ad471462dbeef1b1230218d6a)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:37.318795 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c702b-1fe2-4dd7-855e-09fd4896b038 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:37.403272 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-3e6c702b-1fe2-4dd7-855e-09fd4896b038 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e737c9ad471462dbeef1b1230218d6a'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:37.422646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-3e6c702b-1fe2-4dd7-855e-09fd4896b038 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e737c9ad471462dbeef1b1230218d6a'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:37.434619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-3e6c702b-1fe2-4dd7-855e-09fd4896b038 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e737c9ad471462dbeef1b1230218d6a'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:37.445517 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-3e6c702b-1fe2-4dd7-855e-09fd4896b038 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e737c9ad471462dbeef1b1230218d6a'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:37.446392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-3e6c702b-1fe2-4dd7-855e-09fd4896b038 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e737c9ad471462dbeef1b1230218d6a'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:37.453469 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1321/2640] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:21:37 2022] DELETE /identity/v3/users/0e737c9ad471462dbeef1b1230218d6a => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:38.105294 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ee044-206e-42e0-b6a9-1056b3ed58b8 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:38.107296 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9f6ee044-206e-42e0-b6a9-1056b3ed58b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:38.123250 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ee044-206e-42e0-b6a9-1056b3ed58b8 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:38.237388 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ab4dd-206b-4ead-b0b9-ec5e2279d464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:38.238219 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ab4dd-206b-4ead-b0b9-ec5e2279d464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:38.238545 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ab4dd-206b-4ead-b0b9-ec5e2279d464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:38.238902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ab4dd-206b-4ead-b0b9-ec5e2279d464 admin admin] PATH_INFO: `/v3/projects/472d1ff4ac2a4e098cd7a16208f28ca0` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:38.241791 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ab4dd-206b-4ead-b0b9-ec5e2279d464 admin admin] RBAC: Authorizing `identity:delete_project(project_id=472d1ff4ac2a4e098cd7a16208f28ca0)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:38.246950 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ab4dd-206b-4ead-b0b9-ec5e2279d464 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:38.328508 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-fb4ab4dd-206b-4ead-b0b9-ec5e2279d464 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '472d1ff4ac2a4e098cd7a16208f28ca0', 'request_id': 'req-fb4ab4dd-206b-4ead-b0b9-ec5e2279d464'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:38.341581 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-fb4ab4dd-206b-4ead-b0b9-ec5e2279d464 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '472d1ff4ac2a4e098cd7a16208f28ca0', 'request_id': 'req-fb4ab4dd-206b-4ead-b0b9-ec5e2279d464'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:38.353223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-fb4ab4dd-206b-4ead-b0b9-ec5e2279d464 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '472d1ff4ac2a4e098cd7a16208f28ca0', 'request_id': 'req-fb4ab4dd-206b-4ead-b0b9-ec5e2279d464'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:38.378160 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-fb4ab4dd-206b-4ead-b0b9-ec5e2279d464 admin admin] The token cache is being invalidate because project 472d1ff4ac2a4e098cd7a16208f28ca0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:21:38.379074 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-fb4ab4dd-206b-4ead-b0b9-ec5e2279d464 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:38.379243 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-fb4ab4dd-206b-4ead-b0b9-ec5e2279d464 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:38.380670 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1320/2641] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:21:38 2022] DELETE /identity/v3/projects/472d1ff4ac2a4e098cd7a16208f28ca0 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:38.387335 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6ddd6-2264-4928-ac67-1b92f9b553f5 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:38.388339 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6ddd6-2264-4928-ac67-1b92f9b553f5 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:38.388721 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6ddd6-2264-4928-ac67-1b92f9b553f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:38.417283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-55c6ddd6-2264-4928-ac67-1b92f9b553f5 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:38.418900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-55c6ddd6-2264-4928-ac67-1b92f9b553f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:38.521126 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1322/2642] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:21:38.537228 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8528e2a-765f-446d-b435-920ee79dc56b None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:38.537922 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8528e2a-765f-446d-b435-920ee79dc56b None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:38.538354 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8528e2a-765f-446d-b435-920ee79dc56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:38.571968 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-d8528e2a-765f-446d-b435-920ee79dc56b None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:38.574289 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d8528e2a-765f-446d-b435-920ee79dc56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:38.591253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1321/2643] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:21:38.592994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6ddd6-2264-4928-ac67-1b92f9b553f5 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:38.608908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6ddd6-2264-4928-ac67-1b92f9b553f5 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:38.618383 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10255bb1-0846-445f-8238-818a7b5bdb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:38.619305 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10255bb1-0846-445f-8238-818a7b5bdb94 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:38.619741 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10255bb1-0846-445f-8238-818a7b5bdb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:38.620164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10255bb1-0846-445f-8238-818a7b5bdb94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:38.621187 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10255bb1-0846-445f-8238-818a7b5bdb94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:38.621662 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10255bb1-0846-445f-8238-818a7b5bdb94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:38.624434 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10255bb1-0846-445f-8238-818a7b5bdb94 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:38.629368 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:38.657162 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1323/2644] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:38.659649 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8528e2a-765f-446d-b435-920ee79dc56b None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:38.672689 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8528e2a-765f-446d-b435-920ee79dc56b None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:38.685597 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef323d-a34b-482b-9121-5f04f8d5a3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:38.686316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef323d-a34b-482b-9121-5f04f8d5a3b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:38.686630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef323d-a34b-482b-9121-5f04f8d5a3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:38.686910 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef323d-a34b-482b-9121-5f04f8d5a3b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:38.687637 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef323d-a34b-482b-9121-5f04f8d5a3b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:38.692015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef323d-a34b-482b-9121-5f04f8d5a3b2 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:38.721463 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:38.738188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1322/2645] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:21:38 2022] POST /identity/v3/projects => generated 365 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:38.745058 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10255bb1-0846-445f-8238-818a7b5bdb94 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:38.768916 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10255bb1-0846-445f-8238-818a7b5bdb94 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:38.861059 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb108f-9e41-48ac-aaff-3d5dc893eaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:38.862183 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb108f-9e41-48ac-aaff-3d5dc893eaf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:38.862679 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb108f-9e41-48ac-aaff-3d5dc893eaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:38.863251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb108f-9e41-48ac-aaff-3d5dc893eaf7 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:38.864322 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb108f-9e41-48ac-aaff-3d5dc893eaf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:38.867616 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb108f-9e41-48ac-aaff-3d5dc893eaf7 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:38.905159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1324/2646] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:21:38 2022] POST /identity/v3/users => generated 283 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:38.911349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef323d-a34b-482b-9121-5f04f8d5a3b2 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:38.931005 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef323d-a34b-482b-9121-5f04f8d5a3b2 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:38.942251 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7956262-c986-43f5-8449-1ea52f5b7911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:38.944613 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7956262-c986-43f5-8449-1ea52f5b7911 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:38.945139 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7956262-c986-43f5-8449-1ea52f5b7911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:38.945894 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7956262-c986-43f5-8449-1ea52f5b7911 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:38.947058 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7956262-c986-43f5-8449-1ea52f5b7911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:38.950982 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7956262-c986-43f5-8449-1ea52f5b7911 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:38.972936 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1323/2647] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:21:38.974779 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb108f-9e41-48ac-aaff-3d5dc893eaf7 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:38.989442 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb108f-9e41-48ac-aaff-3d5dc893eaf7 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:38.997450 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560eb26c-0ca0-44b0-89e5-155db6647033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:38.998533 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560eb26c-0ca0-44b0-89e5-155db6647033 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:38.999093 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560eb26c-0ca0-44b0-89e5-155db6647033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:38.999708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560eb26c-0ca0-44b0-89e5-155db6647033 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:39.000505 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560eb26c-0ca0-44b0-89e5-155db6647033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:39.002278 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560eb26c-0ca0-44b0-89e5-155db6647033 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:39.020706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1325/2648] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:21:39.026603 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7956262-c986-43f5-8449-1ea52f5b7911 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:39.043788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7956262-c986-43f5-8449-1ea52f5b7911 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:39.059939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb8e53-b883-4bce-abb4-e928be646a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:39.060759 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb8e53-b883-4bce-abb4-e928be646a83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:39.062577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb8e53-b883-4bce-abb4-e928be646a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:39.062932 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb8e53-b883-4bce-abb4-e928be646a83 admin admin] PATH_INFO: `/v3/projects/f9cd0f42641c4f27a48b383fd1e4ab6f/users/82bc33ae68724d56b51f5c7df29e1f63/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:39.097563 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb8e53-b883-4bce-abb4-e928be646a83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9cd0f42641c4f27a48b383fd1e4ab6f, user_id=82bc33ae68724d56b51f5c7df29e1f63, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:39.102470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb8e53-b883-4bce-abb4-e928be646a83 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:39.132078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1324/2649] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:21:39 2022] PUT /identity/v3/projects/f9cd0f42641c4f27a48b383fd1e4ab6f/users/82bc33ae68724d56b51f5c7df29e1f63/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:39.144191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a058b-364e-4725-a0c2-7ee8354e0d6d None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:39.145209 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a058b-364e-4725-a0c2-7ee8354e0d6d None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:39.145665 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a058b-364e-4725-a0c2-7ee8354e0d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:39.194780 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-a44a058b-364e-4725-a0c2-7ee8354e0d6d None None] MFA Rules not processed for user `82bc33ae68724d56b51f5c7df29e1f63`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:39.196916 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a44a058b-364e-4725-a0c2-7ee8354e0d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:39.348936 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1326/2650] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:39.355065 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb8e53-b883-4bce-abb4-e928be646a83 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:39.479338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb8e53-b883-4bce-abb4-e928be646a83 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:39.488464 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc735ed-020c-4fb5-9ee4-28934f8d7e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:39.489343 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc735ed-020c-4fb5-9ee4-28934f8d7e85 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:39.491035 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc735ed-020c-4fb5-9ee4-28934f8d7e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:39.491035 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc735ed-020c-4fb5-9ee4-28934f8d7e85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:39.491705 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc735ed-020c-4fb5-9ee4-28934f8d7e85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:39.494413 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc735ed-020c-4fb5-9ee4-28934f8d7e85 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:39.554664 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1325/2651] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:21:39 2022] POST /identity/v3/projects => generated 365 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:39.556129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a058b-364e-4725-a0c2-7ee8354e0d6d None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:39.603964 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a058b-364e-4725-a0c2-7ee8354e0d6d None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:39.712384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22ae8f-4d9f-4a22-93cd-7e0fb96ee2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:39.713880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22ae8f-4d9f-4a22-93cd-7e0fb96ee2b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:39.713880 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22ae8f-4d9f-4a22-93cd-7e0fb96ee2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:39.714191 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22ae8f-4d9f-4a22-93cd-7e0fb96ee2b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:39.714979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22ae8f-4d9f-4a22-93cd-7e0fb96ee2b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:39.717399 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22ae8f-4d9f-4a22-93cd-7e0fb96ee2b2 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:39.744232 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:39.772477 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1327/2652] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:21:39 2022] POST /identity/v3/users => generated 289 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:21:39.779060 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc735ed-020c-4fb5-9ee4-28934f8d7e85 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:39.801629 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc735ed-020c-4fb5-9ee4-28934f8d7e85 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:39.815104 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5003e-9a70-4de6-9854-4ba24d8e9777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:39.815841 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5003e-9a70-4de6-9854-4ba24d8e9777 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:39.816219 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5003e-9a70-4de6-9854-4ba24d8e9777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:39.816643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5003e-9a70-4de6-9854-4ba24d8e9777 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:39.819087 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5003e-9a70-4de6-9854-4ba24d8e9777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:39.823024 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5003e-9a70-4de6-9854-4ba24d8e9777 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:39.842386 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1326/2653] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:21:39.847709 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22ae8f-4d9f-4a22-93cd-7e0fb96ee2b2 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:39.867073 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22ae8f-4d9f-4a22-93cd-7e0fb96ee2b2 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:39.876555 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4254b4-a0bb-419a-b274-81f13b999085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:39.877802 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4254b4-a0bb-419a-b274-81f13b999085 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:39.878293 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4254b4-a0bb-419a-b274-81f13b999085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:39.878784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4254b4-a0bb-419a-b274-81f13b999085 admin admin] PATH_INFO: `/v3/projects/0556a89fc7b24114815cc6cbadbc04cc/users/123f0794aafa48fbbb62ab73cbb3f696/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:39.906564 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4254b4-a0bb-419a-b274-81f13b999085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0556a89fc7b24114815cc6cbadbc04cc, user_id=123f0794aafa48fbbb62ab73cbb3f696, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:39.909462 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4254b4-a0bb-419a-b274-81f13b999085 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:39.941779 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1328/2654] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:21:39 2022] PUT /identity/v3/projects/0556a89fc7b24114815cc6cbadbc04cc/users/123f0794aafa48fbbb62ab73cbb3f696/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:39.947194 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5003e-9a70-4de6-9854-4ba24d8e9777 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:39.969941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5003e-9a70-4de6-9854-4ba24d8e9777 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:40.090593 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3d33e-de70-4763-8fb3-f959159714f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:40.091613 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3d33e-de70-4763-8fb3-f959159714f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:40.092096 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3d33e-de70-4763-8fb3-f959159714f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:40.092593 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3d33e-de70-4763-8fb3-f959159714f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:40.093491 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c3d33e-de70-4763-8fb3-f959159714f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:40.095468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c3d33e-de70-4763-8fb3-f959159714f9 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:40.110254 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1327/2655] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:21:40.116536 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4254b4-a0bb-419a-b274-81f13b999085 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:40.130560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4254b4-a0bb-419a-b274-81f13b999085 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:40.145066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03311b8-7a38-4aae-9643-d50c74e4c0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:40.145923 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03311b8-7a38-4aae-9643-d50c74e4c0a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:40.146301 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03311b8-7a38-4aae-9643-d50c74e4c0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:40.146901 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03311b8-7a38-4aae-9643-d50c74e4c0a9 admin admin] PATH_INFO: `/v3/projects/0556a89fc7b24114815cc6cbadbc04cc/users/123f0794aafa48fbbb62ab73cbb3f696/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:40.149391 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03311b8-7a38-4aae-9643-d50c74e4c0a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0556a89fc7b24114815cc6cbadbc04cc, user_id=123f0794aafa48fbbb62ab73cbb3f696, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:40.152394 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03311b8-7a38-4aae-9643-d50c74e4c0a9 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:40.169349 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1329/2656] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:21:40 2022] PUT /identity/v3/projects/0556a89fc7b24114815cc6cbadbc04cc/users/123f0794aafa48fbbb62ab73cbb3f696/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:40.185123 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13261de1-6c43-4db5-8d86-802ebfabba5f None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:40.185991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13261de1-6c43-4db5-8d86-802ebfabba5f None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:40.185991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13261de1-6c43-4db5-8d86-802ebfabba5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:40.234950 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-13261de1-6c43-4db5-8d86-802ebfabba5f None None] MFA Rules not processed for user `123f0794aafa48fbbb62ab73cbb3f696`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:40.237764 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-13261de1-6c43-4db5-8d86-802ebfabba5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:40.275524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:40.379520 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1328/2657] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:40.404753 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03311b8-7a38-4aae-9643-d50c74e4c0a9 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:40.407283 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c03311b8-7a38-4aae-9643-d50c74e4c0a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:40.546397 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03311b8-7a38-4aae-9643-d50c74e4c0a9 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:40.551681 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec3c9d-048f-4785-81cd-bb7fe4fb22c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:40.553617 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec3c9d-048f-4785-81cd-bb7fe4fb22c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:40.554290 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec3c9d-048f-4785-81cd-bb7fe4fb22c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:40.554997 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec3c9d-048f-4785-81cd-bb7fe4fb22c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:40.575736 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec3c9d-048f-4785-81cd-bb7fe4fb22c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:40.581016 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec3c9d-048f-4785-81cd-bb7fe4fb22c8 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:40.618223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1330/2658] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:40 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:21:40.775623 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13261de1-6c43-4db5-8d86-802ebfabba5f None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:40.796802 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13261de1-6c43-4db5-8d86-802ebfabba5f None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:40.803992 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0a841-b106-4f68-983b-8f9ce000d8ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:40.805109 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0a841-b106-4f68-983b-8f9ce000d8ef service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:40.805742 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0a841-b106-4f68-983b-8f9ce000d8ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:40.806538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0a841-b106-4f68-983b-8f9ce000d8ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:40.954129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c0a841-b106-4f68-983b-8f9ce000d8ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:40.961359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c0a841-b106-4f68-983b-8f9ce000d8ef service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:40.975781 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1329/2659] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:43.470018 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec3c9d-048f-4785-81cd-bb7fe4fb22c8 service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:43.471831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-98ec3c9d-048f-4785-81cd-bb7fe4fb22c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:43.544447 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec3c9d-048f-4785-81cd-bb7fe4fb22c8 service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:43.550084 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ac7f5-5323-4bbe-a9a2-b308489a658b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:43.550954 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ac7f5-5323-4bbe-a9a2-b308489a658b service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:43.551360 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ac7f5-5323-4bbe-a9a2-b308489a658b service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:43.551789 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ac7f5-5323-4bbe-a9a2-b308489a658b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:43.554456 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c76ac7f5-5323-4bbe-a9a2-b308489a658b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:43.634295 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ac7f5-5323-4bbe-a9a2-b308489a658b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:43.635881 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ac7f5-5323-4bbe-a9a2-b308489a658b service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:43.651438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1331/2660] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:46.732653 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0a841-b106-4f68-983b-8f9ce000d8ef service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:46.745645 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0a841-b106-4f68-983b-8f9ce000d8ef service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:46.750677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7896607-0a71-407e-9fd4-e066aa254d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:46.751317 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7896607-0a71-407e-9fd4-e066aa254d4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:46.751601 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7896607-0a71-407e-9fd4-e066aa254d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:46.751874 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7896607-0a71-407e-9fd4-e066aa254d4e admin admin] PATH_INFO: `/v3/users/82bc33ae68724d56b51f5c7df29e1f63` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:46.753034 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7896607-0a71-407e-9fd4-e066aa254d4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=82bc33ae68724d56b51f5c7df29e1f63)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:46.755228 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7896607-0a71-407e-9fd4-e066aa254d4e admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:46.826853 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a7896607-0a71-407e-9fd4-e066aa254d4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82bc33ae68724d56b51f5c7df29e1f63'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:46.836336 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a7896607-0a71-407e-9fd4-e066aa254d4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82bc33ae68724d56b51f5c7df29e1f63'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:46.859451 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a7896607-0a71-407e-9fd4-e066aa254d4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82bc33ae68724d56b51f5c7df29e1f63'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:46.869423 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a7896607-0a71-407e-9fd4-e066aa254d4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82bc33ae68724d56b51f5c7df29e1f63'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:46.870410 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a7896607-0a71-407e-9fd4-e066aa254d4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82bc33ae68724d56b51f5c7df29e1f63'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:46.879732 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1330/2661] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:21:46 2022] DELETE /identity/v3/users/82bc33ae68724d56b51f5c7df29e1f63 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:46.884478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c97b39-d0fd-4cab-a122-52cf42f2eb98 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:46.885323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c97b39-d0fd-4cab-a122-52cf42f2eb98 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:46.885879 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c97b39-d0fd-4cab-a122-52cf42f2eb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:46.909832 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-23c97b39-d0fd-4cab-a122-52cf42f2eb98 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:46.911935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-23c97b39-d0fd-4cab-a122-52cf42f2eb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:46.918776 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:47.018047 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1332/2662] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:47.030253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7896607-0a71-407e-9fd4-e066aa254d4e admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:47.034340 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-a7896607-0a71-407e-9fd4-e066aa254d4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:47.082197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:47.145672 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7896607-0a71-407e-9fd4-e066aa254d4e admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:47.150334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5ff54-2753-4333-af2e-76aeec889440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:47.150982 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5ff54-2753-4333-af2e-76aeec889440 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:47.151219 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5ff54-2753-4333-af2e-76aeec889440 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:47.151538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5ff54-2753-4333-af2e-76aeec889440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:47.167299 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d5ff54-2753-4333-af2e-76aeec889440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:47.170469 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d5ff54-2753-4333-af2e-76aeec889440 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:47.185390 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1331/2663] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:21:47.555222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c97b39-d0fd-4cab-a122-52cf42f2eb98 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:47.556938 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-23c97b39-d0fd-4cab-a122-52cf42f2eb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:47.596758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c97b39-d0fd-4cab-a122-52cf42f2eb98 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:47.619321 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c24c4-9cb1-45cd-a5cd-4ddab3c36cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:47.621632 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c24c4-9cb1-45cd-a5cd-4ddab3c36cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:47.623808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c24c4-9cb1-45cd-a5cd-4ddab3c36cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:47.625389 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c24c4-9cb1-45cd-a5cd-4ddab3c36cd5 admin admin] PATH_INFO: `/v3/projects/f9cd0f42641c4f27a48b383fd1e4ab6f` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:47.626982 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c24c4-9cb1-45cd-a5cd-4ddab3c36cd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9cd0f42641c4f27a48b383fd1e4ab6f)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:47.630566 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c24c4-9cb1-45cd-a5cd-4ddab3c36cd5 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:47.754050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-647c24c4-9cb1-45cd-a5cd-4ddab3c36cd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9cd0f42641c4f27a48b383fd1e4ab6f', 'request_id': 'req-647c24c4-9cb1-45cd-a5cd-4ddab3c36cd5'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:47.768424 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-647c24c4-9cb1-45cd-a5cd-4ddab3c36cd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9cd0f42641c4f27a48b383fd1e4ab6f', 'request_id': 'req-647c24c4-9cb1-45cd-a5cd-4ddab3c36cd5'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:47.801571 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-647c24c4-9cb1-45cd-a5cd-4ddab3c36cd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9cd0f42641c4f27a48b383fd1e4ab6f', 'request_id': 'req-647c24c4-9cb1-45cd-a5cd-4ddab3c36cd5'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:47.816635 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-647c24c4-9cb1-45cd-a5cd-4ddab3c36cd5 admin admin] The token cache is being invalidate because project f9cd0f42641c4f27a48b383fd1e4ab6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:21:47.817256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-647c24c4-9cb1-45cd-a5cd-4ddab3c36cd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:47.818296 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-647c24c4-9cb1-45cd-a5cd-4ddab3c36cd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:47.820397 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1333/2664] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:21:47 2022] DELETE /identity/v3/projects/f9cd0f42641c4f27a48b383fd1e4ab6f => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:47.825033 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5ff54-2753-4333-af2e-76aeec889440 service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:47.839937 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5ff54-2753-4333-af2e-76aeec889440 service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:47.926120 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7267d5c-9a17-4baa-8bcd-a8753300bd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:47.927049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7267d5c-9a17-4baa-8bcd-a8753300bd34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:47.928056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7267d5c-9a17-4baa-8bcd-a8753300bd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:47.928631 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7267d5c-9a17-4baa-8bcd-a8753300bd34 admin admin] PATH_INFO: `/v3/users/123f0794aafa48fbbb62ab73cbb3f696` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:47.930805 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7267d5c-9a17-4baa-8bcd-a8753300bd34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=123f0794aafa48fbbb62ab73cbb3f696)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:47.933101 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7267d5c-9a17-4baa-8bcd-a8753300bd34 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:47.991114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7ef62-e53c-477e-a6f7-0743d4f41430 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:47.992715 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7ef62-e53c-477e-a6f7-0743d4f41430 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:47.993240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7ef62-e53c-477e-a6f7-0743d4f41430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:48.014298 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c7267d5c-9a17-4baa-8bcd-a8753300bd34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '123f0794aafa48fbbb62ab73cbb3f696'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:48.020949 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-4dc7ef62-e53c-477e-a6f7-0743d4f41430 None None] MFA Rules not processed for user `c7c8422c3d314adca25acf967ffd0998`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:48.022559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-4dc7ef62-e53c-477e-a6f7-0743d4f41430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:48.024492 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c7267d5c-9a17-4baa-8bcd-a8753300bd34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '123f0794aafa48fbbb62ab73cbb3f696'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:48.056404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c7267d5c-9a17-4baa-8bcd-a8753300bd34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '123f0794aafa48fbbb62ab73cbb3f696'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:48.072174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c7267d5c-9a17-4baa-8bcd-a8753300bd34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '123f0794aafa48fbbb62ab73cbb3f696'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:48.073684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c7267d5c-9a17-4baa-8bcd-a8753300bd34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '123f0794aafa48fbbb62ab73cbb3f696'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:48.085223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1332/2665] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:21:47 2022] DELETE /identity/v3/users/123f0794aafa48fbbb62ab73cbb3f696 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:48.168267 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1334/2666] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:48.587515 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7267d5c-9a17-4baa-8bcd-a8753300bd34 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:48.589602 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c7267d5c-9a17-4baa-8bcd-a8753300bd34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:48.688460 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:48.718728 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7267d5c-9a17-4baa-8bcd-a8753300bd34 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:48.735475 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c56fb-1a4a-43e7-9937-9dbb3069629f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:48.736843 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c56fb-1a4a-43e7-9937-9dbb3069629f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:48.738398 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c56fb-1a4a-43e7-9937-9dbb3069629f admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:48.738398 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c56fb-1a4a-43e7-9937-9dbb3069629f admin admin] PATH_INFO: `/v3/projects/0556a89fc7b24114815cc6cbadbc04cc` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:48.740636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c56fb-1a4a-43e7-9937-9dbb3069629f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0556a89fc7b24114815cc6cbadbc04cc)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:48.743337 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c56fb-1a4a-43e7-9937-9dbb3069629f admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:48.817167 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-ca5c56fb-1a4a-43e7-9937-9dbb3069629f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0556a89fc7b24114815cc6cbadbc04cc', 'request_id': 'req-ca5c56fb-1a4a-43e7-9937-9dbb3069629f'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:48.824818 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-ca5c56fb-1a4a-43e7-9937-9dbb3069629f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0556a89fc7b24114815cc6cbadbc04cc', 'request_id': 'req-ca5c56fb-1a4a-43e7-9937-9dbb3069629f'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:48.831322 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-ca5c56fb-1a4a-43e7-9937-9dbb3069629f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0556a89fc7b24114815cc6cbadbc04cc', 'request_id': 'req-ca5c56fb-1a4a-43e7-9937-9dbb3069629f'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:48.853150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-ca5c56fb-1a4a-43e7-9937-9dbb3069629f admin admin] The token cache is being invalidate because project 0556a89fc7b24114815cc6cbadbc04cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:21:48.853453 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-ca5c56fb-1a4a-43e7-9937-9dbb3069629f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:48.854190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-ca5c56fb-1a4a-43e7-9937-9dbb3069629f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:48.859899 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1333/2667] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:21:48 2022] DELETE /identity/v3/projects/0556a89fc7b24114815cc6cbadbc04cc => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:48.862140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654bb1d-af10-4213-8b81-e53270a98e9c None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:48.863165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654bb1d-af10-4213-8b81-e53270a98e9c None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:48.863165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654bb1d-af10-4213-8b81-e53270a98e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:48.889760 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-9654bb1d-af10-4213-8b81-e53270a98e9c None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:48.890891 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-9654bb1d-af10-4213-8b81-e53270a98e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:48.942255 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:48.991657 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1335/2668] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:49.001550 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa55b7-8f84-428e-a70a-56513623fbed None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:49.001947 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa55b7-8f84-428e-a70a-56513623fbed None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:49.002330 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa55b7-8f84-428e-a70a-56513623fbed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:49.027979 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-ebfa55b7-8f84-428e-a70a-56513623fbed None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:49.029240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-ebfa55b7-8f84-428e-a70a-56513623fbed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:49.038461 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1334/2669] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:49.043620 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654bb1d-af10-4213-8b81-e53270a98e9c None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:49.056459 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654bb1d-af10-4213-8b81-e53270a98e9c None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:49.065008 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7991a-d786-411e-9be0-ab0aec20d63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:49.066075 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7991a-d786-411e-9be0-ab0aec20d63c admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:49.066379 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7991a-d786-411e-9be0-ab0aec20d63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:49.066712 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7991a-d786-411e-9be0-ab0aec20d63c admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:49.067448 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7991a-d786-411e-9be0-ab0aec20d63c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:49.067845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7991a-d786-411e-9be0-ab0aec20d63c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:49.070318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7991a-d786-411e-9be0-ab0aec20d63c admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:49.098062 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1336/2670] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:49.100286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa55b7-8f84-428e-a70a-56513623fbed None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:49.112074 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa55b7-8f84-428e-a70a-56513623fbed None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:49.119196 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c353a71-5695-4c26-985b-dbdc64a2a249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:49.119959 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c353a71-5695-4c26-985b-dbdc64a2a249 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:49.120262 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c353a71-5695-4c26-985b-dbdc64a2a249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:49.120599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c353a71-5695-4c26-985b-dbdc64a2a249 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:49.121310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c353a71-5695-4c26-985b-dbdc64a2a249 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:49.123549 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c353a71-5695-4c26-985b-dbdc64a2a249 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:49.161979 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1335/2671] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:21:49 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:21:49.163615 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7991a-d786-411e-9be0-ab0aec20d63c admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:49.178193 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7991a-d786-411e-9be0-ab0aec20d63c admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:49.292875 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5bd53b-59c8-4e9c-bbcb-aa43a10ef5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:49.293811 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bd53b-59c8-4e9c-bbcb-aa43a10ef5f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:49.294188 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bd53b-59c8-4e9c-bbcb-aa43a10ef5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:49.294600 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bd53b-59c8-4e9c-bbcb-aa43a10ef5f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:49.295563 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5bd53b-59c8-4e9c-bbcb-aa43a10ef5f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:49.297597 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5bd53b-59c8-4e9c-bbcb-aa43a10ef5f9 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:49.334252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1337/2672] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:21:49 2022] POST /identity/v3/users => generated 287 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:49.346425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c353a71-5695-4c26-985b-dbdc64a2a249 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:49.377533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c353a71-5695-4c26-985b-dbdc64a2a249 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:49.386279 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1c08c-0a44-470f-a74d-c3b5b079fac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:49.387510 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1c08c-0a44-470f-a74d-c3b5b079fac7 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:49.388003 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1c08c-0a44-470f-a74d-c3b5b079fac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:49.388480 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1c08c-0a44-470f-a74d-c3b5b079fac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:49.389657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b1c08c-0a44-470f-a74d-c3b5b079fac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:49.392553 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b1c08c-0a44-470f-a74d-c3b5b079fac7 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:49.414833 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1336/2673] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:21:49.419093 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5bd53b-59c8-4e9c-bbcb-aa43a10ef5f9 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:49.445733 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5bd53b-59c8-4e9c-bbcb-aa43a10ef5f9 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:49.451610 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac5f36-cd22-450e-adc7-78a7d12997f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:49.452281 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac5f36-cd22-450e-adc7-78a7d12997f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:49.452561 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac5f36-cd22-450e-adc7-78a7d12997f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:49.452912 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac5f36-cd22-450e-adc7-78a7d12997f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:49.453803 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac5f36-cd22-450e-adc7-78a7d12997f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:49.455510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac5f36-cd22-450e-adc7-78a7d12997f7 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:49.472723 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1338/2674] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:21:49.477621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1c08c-0a44-470f-a74d-c3b5b079fac7 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:49.493263 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1c08c-0a44-470f-a74d-c3b5b079fac7 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:49.500052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d7f5d-8f1f-4a87-be2a-70d7f5086c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:49.500899 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d7f5d-8f1f-4a87-be2a-70d7f5086c5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:49.501809 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d7f5d-8f1f-4a87-be2a-70d7f5086c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:49.502140 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d7f5d-8f1f-4a87-be2a-70d7f5086c5c admin admin] PATH_INFO: `/v3/projects/cd7e0517ae4f43029777eeb41f21ebe7/users/6ade8b5c698f437b96a2385e1218cbe9/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:49.531770 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d7f5d-8f1f-4a87-be2a-70d7f5086c5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd7e0517ae4f43029777eeb41f21ebe7, user_id=6ade8b5c698f437b96a2385e1218cbe9, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:49.549417 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d7f5d-8f1f-4a87-be2a-70d7f5086c5c admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:49.597150 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1337/2675] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:21:49 2022] PUT /identity/v3/projects/cd7e0517ae4f43029777eeb41f21ebe7/users/6ade8b5c698f437b96a2385e1218cbe9/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:49.698176 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1651bb-212e-4977-ae43-6bb856655fa0 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:49.699171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1651bb-212e-4977-ae43-6bb856655fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:49.699724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1651bb-212e-4977-ae43-6bb856655fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:49.762021 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-2a1651bb-212e-4977-ae43-6bb856655fa0 None None] MFA Rules not processed for user `6ade8b5c698f437b96a2385e1218cbe9`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:49.764402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-2a1651bb-212e-4977-ae43-6bb856655fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:49.908280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1339/2676] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:49.924668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d7f5d-8f1f-4a87-be2a-70d7f5086c5c admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:49.929186 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-385d7f5d-8f1f-4a87-be2a-70d7f5086c5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:50.026330 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d7f5d-8f1f-4a87-be2a-70d7f5086c5c admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:50.030407 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc40cb-a958-47a4-b202-318fbeaaf206 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:50.031214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc40cb-a958-47a4-b202-318fbeaaf206 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:50.031438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc40cb-a958-47a4-b202-318fbeaaf206 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:50.031715 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc40cb-a958-47a4-b202-318fbeaaf206 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:50.043470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc40cb-a958-47a4-b202-318fbeaaf206 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:50.045371 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc40cb-a958-47a4-b202-318fbeaaf206 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:50.065806 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1338/2677] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:53.689355 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1651bb-212e-4977-ae43-6bb856655fa0 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:53.760752 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:53.807108 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1651bb-212e-4977-ae43-6bb856655fa0 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:53.813181 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19836a-121a-4fe6-9201-9ec4975429c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:53.814113 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19836a-121a-4fe6-9201-9ec4975429c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:53.814476 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19836a-121a-4fe6-9201-9ec4975429c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:53.814827 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19836a-121a-4fe6-9201-9ec4975429c3 admin admin] PATH_INFO: `/v3/users/6ade8b5c698f437b96a2385e1218cbe9` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:53.817172 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19836a-121a-4fe6-9201-9ec4975429c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ade8b5c698f437b96a2385e1218cbe9)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:53.819929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19836a-121a-4fe6-9201-9ec4975429c3 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:53.901619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-aa19836a-121a-4fe6-9201-9ec4975429c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ade8b5c698f437b96a2385e1218cbe9'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:53.918579 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-aa19836a-121a-4fe6-9201-9ec4975429c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ade8b5c698f437b96a2385e1218cbe9'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:53.931642 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-aa19836a-121a-4fe6-9201-9ec4975429c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ade8b5c698f437b96a2385e1218cbe9'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:53.942132 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-aa19836a-121a-4fe6-9201-9ec4975429c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ade8b5c698f437b96a2385e1218cbe9'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:53.943067 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-aa19836a-121a-4fe6-9201-9ec4975429c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ade8b5c698f437b96a2385e1218cbe9'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:53.951769 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1340/2678] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:21:53 2022] DELETE /identity/v3/users/6ade8b5c698f437b96a2385e1218cbe9 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:53.960695 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27eea19-186d-44f3-9305-7cf928cb194d None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:53.963348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27eea19-186d-44f3-9305-7cf928cb194d None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:53.964222 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27eea19-186d-44f3-9305-7cf928cb194d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:53.991411 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-f27eea19-186d-44f3-9305-7cf928cb194d None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:53.993441 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-f27eea19-186d-44f3-9305-7cf928cb194d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:54.014368 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:54.115434 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1339/2679] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:54.128850 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19836a-121a-4fe6-9201-9ec4975429c3 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:54.132222 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-aa19836a-121a-4fe6-9201-9ec4975429c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:54.227958 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19836a-121a-4fe6-9201-9ec4975429c3 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:54.245196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66158dd6-f233-4a60-922e-6d5e4a4a7f2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:54.246572 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66158dd6-f233-4a60-922e-6d5e4a4a7f2e service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:54.246920 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66158dd6-f233-4a60-922e-6d5e4a4a7f2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:54.247271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66158dd6-f233-4a60-922e-6d5e4a4a7f2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:54.261986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66158dd6-f233-4a60-922e-6d5e4a4a7f2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:54.264420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66158dd6-f233-4a60-922e-6d5e4a4a7f2e service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:54.279613 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1341/2680] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:55.212065 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27eea19-186d-44f3-9305-7cf928cb194d None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:55.213660 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-f27eea19-186d-44f3-9305-7cf928cb194d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:55.227164 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27eea19-186d-44f3-9305-7cf928cb194d None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:55.235546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66158dd6-f233-4a60-922e-6d5e4a4a7f2e service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:55.237156 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-66158dd6-f233-4a60-922e-6d5e4a4a7f2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:55.238065 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4229e-5a5d-4982-9822-dd3a8f150985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:55.239134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4229e-5a5d-4982-9822-dd3a8f150985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:55.239849 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4229e-5a5d-4982-9822-dd3a8f150985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:55.240212 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4229e-5a5d-4982-9822-dd3a8f150985 admin admin] PATH_INFO: `/v3/users/70c47e7154fd41a7adb4735a519f1484` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:55.242590 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d4229e-5a5d-4982-9822-dd3a8f150985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70c47e7154fd41a7adb4735a519f1484)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:55.245870 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d4229e-5a5d-4982-9822-dd3a8f150985 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:55.254660 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66158dd6-f233-4a60-922e-6d5e4a4a7f2e service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:55.274832 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c78ba3-1bdf-4fb3-83e9-a63de79bc61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:55.275460 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c78ba3-1bdf-4fb3-83e9-a63de79bc61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:55.275740 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c78ba3-1bdf-4fb3-83e9-a63de79bc61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:55.276011 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c78ba3-1bdf-4fb3-83e9-a63de79bc61e admin admin] PATH_INFO: `/v3/projects/cd7e0517ae4f43029777eeb41f21ebe7` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:55.279706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c78ba3-1bdf-4fb3-83e9-a63de79bc61e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd7e0517ae4f43029777eeb41f21ebe7)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:55.281878 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c78ba3-1bdf-4fb3-83e9-a63de79bc61e admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:55.332415 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f0d4229e-5a5d-4982-9822-dd3a8f150985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70c47e7154fd41a7adb4735a519f1484'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:55.344976 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f0d4229e-5a5d-4982-9822-dd3a8f150985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70c47e7154fd41a7adb4735a519f1484'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:55.353998 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e7c78ba3-1bdf-4fb3-83e9-a63de79bc61e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd7e0517ae4f43029777eeb41f21ebe7', 'request_id': 'req-e7c78ba3-1bdf-4fb3-83e9-a63de79bc61e'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:55.355652 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:55.367094 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e7c78ba3-1bdf-4fb3-83e9-a63de79bc61e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd7e0517ae4f43029777eeb41f21ebe7', 'request_id': 'req-e7c78ba3-1bdf-4fb3-83e9-a63de79bc61e'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:55.378389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f0d4229e-5a5d-4982-9822-dd3a8f150985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70c47e7154fd41a7adb4735a519f1484'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:55.378945 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e7c78ba3-1bdf-4fb3-83e9-a63de79bc61e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd7e0517ae4f43029777eeb41f21ebe7', 'request_id': 'req-e7c78ba3-1bdf-4fb3-83e9-a63de79bc61e'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:55.389571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f0d4229e-5a5d-4982-9822-dd3a8f150985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70c47e7154fd41a7adb4735a519f1484'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:55.391049 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f0d4229e-5a5d-4982-9822-dd3a8f150985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70c47e7154fd41a7adb4735a519f1484'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:55.398132 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e7c78ba3-1bdf-4fb3-83e9-a63de79bc61e admin admin] The token cache is being invalidate because project cd7e0517ae4f43029777eeb41f21ebe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:21:55.398670 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e7c78ba3-1bdf-4fb3-83e9-a63de79bc61e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:55.399546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e7c78ba3-1bdf-4fb3-83e9-a63de79bc61e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:55.400814 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1340/2681] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:21:55 2022] DELETE /identity/v3/users/70c47e7154fd41a7adb4735a519f1484 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:55.401910 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1342/2682] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:21:55 2022] DELETE /identity/v3/projects/cd7e0517ae4f43029777eeb41f21ebe7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:55.408776 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72c80d-b368-4f85-88b6-a54a71165951 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:55.409089 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72c80d-b368-4f85-88b6-a54a71165951 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:55.409403 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72c80d-b368-4f85-88b6-a54a71165951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:55.434736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-ba72c80d-b368-4f85-88b6-a54a71165951 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:55.436413 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-ba72c80d-b368-4f85-88b6-a54a71165951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:55.524026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1341/2683] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:55.531778 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99330d-c501-4736-822d-2ac6d064555e None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:55.532925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99330d-c501-4736-822d-2ac6d064555e None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:55.532925 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99330d-c501-4736-822d-2ac6d064555e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:55.567948 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-5d99330d-c501-4736-822d-2ac6d064555e None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:55.570276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-5d99330d-c501-4736-822d-2ac6d064555e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:55.598135 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1343/2684] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:55.603282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72c80d-b368-4f85-88b6-a54a71165951 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:55.623410 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72c80d-b368-4f85-88b6-a54a71165951 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:55.632517 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452566ca-0767-4a65-a108-797388969553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:55.633545 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452566ca-0767-4a65-a108-797388969553 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:55.634640 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452566ca-0767-4a65-a108-797388969553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:55.634640 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452566ca-0767-4a65-a108-797388969553 admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:55.636854 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452566ca-0767-4a65-a108-797388969553 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:55.637346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452566ca-0767-4a65-a108-797388969553 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:55.640565 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452566ca-0767-4a65-a108-797388969553 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:55.670584 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1342/2685] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:55.679929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99330d-c501-4736-822d-2ac6d064555e None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:55.704665 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99330d-c501-4736-822d-2ac6d064555e None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:55.715196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf50425-2437-4bb7-8e57-8ee515e18961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:55.716527 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf50425-2437-4bb7-8e57-8ee515e18961 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:55.716703 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452566ca-0767-4a65-a108-797388969553 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:55.717344 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf50425-2437-4bb7-8e57-8ee515e18961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:55.718018 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf50425-2437-4bb7-8e57-8ee515e18961 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:55.719174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-452566ca-0767-4a65-a108-797388969553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:55.720167 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf50425-2437-4bb7-8e57-8ee515e18961 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:55.724481 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf50425-2437-4bb7-8e57-8ee515e18961 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:55.738257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452566ca-0767-4a65-a108-797388969553 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:55.762289 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788e7e3-f011-4819-a86b-00b24c3f6b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:55.763777 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788e7e3-f011-4819-a86b-00b24c3f6b5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:55.764140 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788e7e3-f011-4819-a86b-00b24c3f6b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:55.765248 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788e7e3-f011-4819-a86b-00b24c3f6b5e admin admin] PATH_INFO: `/v3/projects/98b4b754e82445a0b65e88f5dbede11b` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:55.768860 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788e7e3-f011-4819-a86b-00b24c3f6b5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=98b4b754e82445a0b65e88f5dbede11b)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:55.771323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788e7e3-f011-4819-a86b-00b24c3f6b5e admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:55.774745 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1344/2686] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:21:55 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:55.781221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf50425-2437-4bb7-8e57-8ee515e18961 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:55.801051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:55.859389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d788e7e3-f011-4819-a86b-00b24c3f6b5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98b4b754e82445a0b65e88f5dbede11b', 'request_id': 'req-d788e7e3-f011-4819-a86b-00b24c3f6b5e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:55.870295 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d788e7e3-f011-4819-a86b-00b24c3f6b5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98b4b754e82445a0b65e88f5dbede11b', 'request_id': 'req-d788e7e3-f011-4819-a86b-00b24c3f6b5e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:55.879716 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d788e7e3-f011-4819-a86b-00b24c3f6b5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98b4b754e82445a0b65e88f5dbede11b', 'request_id': 'req-d788e7e3-f011-4819-a86b-00b24c3f6b5e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:55.887724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf50425-2437-4bb7-8e57-8ee515e18961 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:55.908419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d788e7e3-f011-4819-a86b-00b24c3f6b5e admin admin] The token cache is being invalidate because project 98b4b754e82445a0b65e88f5dbede11b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:21:55.908933 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d788e7e3-f011-4819-a86b-00b24c3f6b5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:55.910085 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-d788e7e3-f011-4819-a86b-00b24c3f6b5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:55.913526 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1343/2687] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:21:55 2022] DELETE /identity/v3/projects/98b4b754e82445a0b65e88f5dbede11b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:21:55.917602 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788e7e3-f011-4819-a86b-00b24c3f6b5e admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:55.919375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d788e7e3-f011-4819-a86b-00b24c3f6b5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:55.978178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76996bc0-912d-4291-bee6-614bf2562f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:55.978944 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76996bc0-912d-4291-bee6-614bf2562f75 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:55.979310 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76996bc0-912d-4291-bee6-614bf2562f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:55.979859 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76996bc0-912d-4291-bee6-614bf2562f75 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:55.980644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76996bc0-912d-4291-bee6-614bf2562f75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:55.982782 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76996bc0-912d-4291-bee6-614bf2562f75 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:56.014161 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788e7e3-f011-4819-a86b-00b24c3f6b5e admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:56.016533 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1345/2688] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:21:55 2022] POST /identity/v3/users => generated 292 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:21:56.025124 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76996bc0-912d-4291-bee6-614bf2562f75 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:56.028376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-76996bc0-912d-4291-bee6-614bf2562f75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:56.030051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc886953-f68e-43f5-a9f5-e8b8d1ed9a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:56.031192 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc886953-f68e-43f5-a9f5-e8b8d1ed9a26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:56.032551 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc886953-f68e-43f5-a9f5-e8b8d1ed9a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:56.032551 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc886953-f68e-43f5-a9f5-e8b8d1ed9a26 admin admin] PATH_INFO: `/v3/users/434966086e7849bca9cce516c259babe` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:56.035447 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc886953-f68e-43f5-a9f5-e8b8d1ed9a26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=434966086e7849bca9cce516c259babe)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:56.039883 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc886953-f68e-43f5-a9f5-e8b8d1ed9a26 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:56.055013 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76996bc0-912d-4291-bee6-614bf2562f75 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:56.063101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e312ea-e364-4ef4-ab6e-2304fdc09b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:56.064218 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e312ea-e364-4ef4-ab6e-2304fdc09b01 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:56.064766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e312ea-e364-4ef4-ab6e-2304fdc09b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:56.065365 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e312ea-e364-4ef4-ab6e-2304fdc09b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:56.066623 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e312ea-e364-4ef4-ab6e-2304fdc09b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:56.071377 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e312ea-e364-4ef4-ab6e-2304fdc09b01 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:56.096979 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1346/2689] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:21:56.104201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e312ea-e364-4ef4-ab6e-2304fdc09b01 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:56.125085 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e312ea-e364-4ef4-ab6e-2304fdc09b01 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:56.139061 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386a4b7-e662-4739-ba0e-a3dfc61d7df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:56.139693 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386a4b7-e662-4739-ba0e-a3dfc61d7df0 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:56.140226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386a4b7-e662-4739-ba0e-a3dfc61d7df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:56.140768 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386a4b7-e662-4739-ba0e-a3dfc61d7df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:56.141920 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386a4b7-e662-4739-ba0e-a3dfc61d7df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:56.143267 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-bc886953-f68e-43f5-a9f5-e8b8d1ed9a26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '434966086e7849bca9cce516c259babe'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:56.144604 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386a4b7-e662-4739-ba0e-a3dfc61d7df0 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:56.163746 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1347/2690] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:21:56.166536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-bc886953-f68e-43f5-a9f5-e8b8d1ed9a26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '434966086e7849bca9cce516c259babe'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:56.170434 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386a4b7-e662-4739-ba0e-a3dfc61d7df0 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:56.190033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386a4b7-e662-4739-ba0e-a3dfc61d7df0 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:56.198375 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-bc886953-f68e-43f5-a9f5-e8b8d1ed9a26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '434966086e7849bca9cce516c259babe'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:56.202424 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80036e-a193-4651-9332-d18ccaa16517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:56.203251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80036e-a193-4651-9332-d18ccaa16517 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:56.203568 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80036e-a193-4651-9332-d18ccaa16517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:56.203881 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80036e-a193-4651-9332-d18ccaa16517 admin admin] PATH_INFO: `/v3/projects/3d6b1e6a93214e0fbe67b7dfb84f1043/users/47d7f801e70d4e08b73a83451fd522b1/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:56.214415 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-bc886953-f68e-43f5-a9f5-e8b8d1ed9a26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '434966086e7849bca9cce516c259babe'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:56.215600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-bc886953-f68e-43f5-a9f5-e8b8d1ed9a26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '434966086e7849bca9cce516c259babe'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:56.225142 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1344/2691] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:21:55 2022] DELETE /identity/v3/users/434966086e7849bca9cce516c259babe => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:56.243972 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80036e-a193-4651-9332-d18ccaa16517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d6b1e6a93214e0fbe67b7dfb84f1043, user_id=47d7f801e70d4e08b73a83451fd522b1, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:56.249730 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80036e-a193-4651-9332-d18ccaa16517 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:56.264622 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:DELETE=3,keystone:SELECT=91,keystone:INSERT=6 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:56.274103 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1348/2692] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:21:56 2022] PUT /identity/v3/projects/3d6b1e6a93214e0fbe67b7dfb84f1043/users/47d7f801e70d4e08b73a83451fd522b1/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:56.280890 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca5168-8dd6-4eb4-8936-3979940cc15b None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:56.281499 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca5168-8dd6-4eb4-8936-3979940cc15b None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:56.281960 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca5168-8dd6-4eb4-8936-3979940cc15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:56.318573 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-83ca5168-8dd6-4eb4-8936-3979940cc15b None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:56.321331 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-83ca5168-8dd6-4eb4-8936-3979940cc15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:56.355033 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:56.445457 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1345/2693] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:56.456443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80036e-a193-4651-9332-d18ccaa16517 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:56.460533 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-6d80036e-a193-4651-9332-d18ccaa16517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:56.562432 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca5168-8dd6-4eb4-8936-3979940cc15b None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:56.564939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-83ca5168-8dd6-4eb4-8936-3979940cc15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:56.586528 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca5168-8dd6-4eb4-8936-3979940cc15b None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:56.588461 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80036e-a193-4651-9332-d18ccaa16517 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:56.593608 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d685c04-31d2-4279-a2ee-45be4368625d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:56.594385 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d685c04-31d2-4279-a2ee-45be4368625d service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:56.594820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d685c04-31d2-4279-a2ee-45be4368625d service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:56.595352 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d685c04-31d2-4279-a2ee-45be4368625d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:56.596436 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1234466-67f0-404c-8238-f3649dcfdf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:56.597591 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1234466-67f0-404c-8238-f3649dcfdf4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:56.598068 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1234466-67f0-404c-8238-f3649dcfdf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:56.598594 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1234466-67f0-404c-8238-f3649dcfdf4e admin admin] PATH_INFO: `/v3/projects/07029b1dc2414781a7590b61f60839ee` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:56.600787 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1234466-67f0-404c-8238-f3649dcfdf4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=07029b1dc2414781a7590b61f60839ee)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:56.604948 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1234466-67f0-404c-8238-f3649dcfdf4e admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:56.620437 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d685c04-31d2-4279-a2ee-45be4368625d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:56.626963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d685c04-31d2-4279-a2ee-45be4368625d service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:56.645741 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1349/2694] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:56.700568 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f1234466-67f0-404c-8238-f3649dcfdf4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07029b1dc2414781a7590b61f60839ee', 'request_id': 'req-f1234466-67f0-404c-8238-f3649dcfdf4e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:56.713627 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f1234466-67f0-404c-8238-f3649dcfdf4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07029b1dc2414781a7590b61f60839ee', 'request_id': 'req-f1234466-67f0-404c-8238-f3649dcfdf4e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:56.722462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f1234466-67f0-404c-8238-f3649dcfdf4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07029b1dc2414781a7590b61f60839ee', 'request_id': 'req-f1234466-67f0-404c-8238-f3649dcfdf4e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:56.738404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f1234466-67f0-404c-8238-f3649dcfdf4e admin admin] The token cache is being invalidate because project 07029b1dc2414781a7590b61f60839ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:21:56.738699 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f1234466-67f0-404c-8238-f3649dcfdf4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:56.739470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f1234466-67f0-404c-8238-f3649dcfdf4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:21:56.741677 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1346/2695] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:21:56 2022] DELETE /identity/v3/projects/07029b1dc2414781a7590b61f60839ee => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:21:57.745249 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d89a1-f76b-4079-bc95-1bce2c09d0e2 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:57.746169 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d89a1-f76b-4079-bc95-1bce2c09d0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:57.746169 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d89a1-f76b-4079-bc95-1bce2c09d0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:57.788866 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-6b9d89a1-f76b-4079-bc95-1bce2c09d0e2 None None] MFA Rules not processed for user `47d7f801e70d4e08b73a83451fd522b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:57.790393 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-6b9d89a1-f76b-4079-bc95-1bce2c09d0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:57.921783 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1350/2696] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:21:57.935893 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1234466-67f0-404c-8238-f3649dcfdf4e admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:57.938327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-f1234466-67f0-404c-8238-f3649dcfdf4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:58.072181 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1234466-67f0-404c-8238-f3649dcfdf4e admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:58.079610 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ee685-8421-4d3f-aa2e-af5338ee0690 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:58.080409 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ee685-8421-4d3f-aa2e-af5338ee0690 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:58.080834 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ee685-8421-4d3f-aa2e-af5338ee0690 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:58.081236 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ee685-8421-4d3f-aa2e-af5338ee0690 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:58.110439 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ee685-8421-4d3f-aa2e-af5338ee0690 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:58.114711 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ee685-8421-4d3f-aa2e-af5338ee0690 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:58.143608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1347/2697] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:59.752115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d60c7-3965-4698-af67-568b6dac4e90 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:59.753289 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d60c7-3965-4698-af67-568b6dac4e90 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:59.753913 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d60c7-3965-4698-af67-568b6dac4e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:59.783584 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-1c3d60c7-3965-4698-af67-568b6dac4e90 None None] MFA Rules not processed for user `47d7f801e70d4e08b73a83451fd522b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:59.785213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-1c3d60c7-3965-4698-af67-568b6dac4e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:59.797077 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1351/2698] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:21:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:22:08.123964 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:09.776474 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=61 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:10.519433 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ee685-8421-4d3f-aa2e-af5338ee0690 service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:10.521517 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-e80ee685-8421-4d3f-aa2e-af5338ee0690 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:10.629129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ee685-8421-4d3f-aa2e-af5338ee0690 service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:10.635286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d5fb2-6d4b-46c5-9f4c-e396ac72463a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:10.636325 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d5fb2-6d4b-46c5-9f4c-e396ac72463a service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:10.636829 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d5fb2-6d4b-46c5-9f4c-e396ac72463a service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:10.637289 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d5fb2-6d4b-46c5-9f4c-e396ac72463a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:10.640168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d67d5fb2-6d4b-46c5-9f4c-e396ac72463a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:10.734086 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d5fb2-6d4b-46c5-9f4c-e396ac72463a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:10.735661 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d5fb2-6d4b-46c5-9f4c-e396ac72463a service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:10.749554 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1348/2699] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:22:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:22:15.312244 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d60c7-3965-4698-af67-568b6dac4e90 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:15.315486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-1c3d60c7-3965-4698-af67-568b6dac4e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:15.425671 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d5fb2-6d4b-46c5-9f4c-e396ac72463a service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:15.428487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d67d5fb2-6d4b-46c5-9f4c-e396ac72463a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:15.460720 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d5fb2-6d4b-46c5-9f4c-e396ac72463a service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:15.468116 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c719c-8183-4881-86a5-2a9f4e03dcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:15.469370 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c719c-8183-4881-86a5-2a9f4e03dcd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:15.469835 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c719c-8183-4881-86a5-2a9f4e03dcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:15.470238 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c719c-8183-4881-86a5-2a9f4e03dcd5 admin admin] PATH_INFO: `/v3/users/1ffc617c50984f83adc3269b118c8a91` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:15.471964 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c719c-8183-4881-86a5-2a9f4e03dcd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ffc617c50984f83adc3269b118c8a91)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:15.477466 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c719c-8183-4881-86a5-2a9f4e03dcd5 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:15.486254 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d60c7-3965-4698-af67-568b6dac4e90 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:15.497073 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb70e9-b36c-4f04-ba1c-d04a9a43b06c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab2e318b2854c46bc7ed2151a44a2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:15.498271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb70e9-b36c-4f04-ba1c-d04a9a43b06c service placement] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:15.498708 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb70e9-b36c-4f04-ba1c-d04a9a43b06c service placement] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:15.499149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb70e9-b36c-4f04-ba1c-d04a9a43b06c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:15.502316 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f8fb70e9-b36c-4f04-ba1c-d04a9a43b06c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:15.526637 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb70e9-b36c-4f04-ba1c-d04a9a43b06c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:15.530649 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb70e9-b36c-4f04-ba1c-d04a9a43b06c service placement] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:15.558179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1352/2700] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:22:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:22:15.567675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b00c719c-8183-4881-86a5-2a9f4e03dcd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ffc617c50984f83adc3269b118c8a91'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:15.587268 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b00c719c-8183-4881-86a5-2a9f4e03dcd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ffc617c50984f83adc3269b118c8a91'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:15.632187 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b00c719c-8183-4881-86a5-2a9f4e03dcd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ffc617c50984f83adc3269b118c8a91'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:15.647718 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b00c719c-8183-4881-86a5-2a9f4e03dcd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ffc617c50984f83adc3269b118c8a91'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:15.656077 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b00c719c-8183-4881-86a5-2a9f4e03dcd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ffc617c50984f83adc3269b118c8a91'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:15.668299 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1349/2701] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:22:15 2022] DELETE /identity/v3/users/1ffc617c50984f83adc3269b118c8a91 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:22:15.940410 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb70e9-b36c-4f04-ba1c-d04a9a43b06c service placement] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:15.964310 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb70e9-b36c-4f04-ba1c-d04a9a43b06c service placement] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:16.061573 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f03f36b-b476-4911-b8c4-3f1b01fa9a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:16.062932 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03f36b-b476-4911-b8c4-3f1b01fa9a2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:16.063375 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03f36b-b476-4911-b8c4-3f1b01fa9a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:16.064022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03f36b-b476-4911-b8c4-3f1b01fa9a2f admin admin] PATH_INFO: `/v3/projects/fc733dcfcf9043feb354598fc2792ff5` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:16.065584 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f03f36b-b476-4911-b8c4-3f1b01fa9a2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc733dcfcf9043feb354598fc2792ff5)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:16.069465 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f03f36b-b476-4911-b8c4-3f1b01fa9a2f admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:16.158090 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-9f03f36b-b476-4911-b8c4-3f1b01fa9a2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc733dcfcf9043feb354598fc2792ff5', 'request_id': 'req-9f03f36b-b476-4911-b8c4-3f1b01fa9a2f'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:16.167923 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-9f03f36b-b476-4911-b8c4-3f1b01fa9a2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc733dcfcf9043feb354598fc2792ff5', 'request_id': 'req-9f03f36b-b476-4911-b8c4-3f1b01fa9a2f'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:16.179448 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-9f03f36b-b476-4911-b8c4-3f1b01fa9a2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc733dcfcf9043feb354598fc2792ff5', 'request_id': 'req-9f03f36b-b476-4911-b8c4-3f1b01fa9a2f'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:16.196933 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-9f03f36b-b476-4911-b8c4-3f1b01fa9a2f admin admin] The token cache is being invalidate because project fc733dcfcf9043feb354598fc2792ff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:22:16.197261 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-9f03f36b-b476-4911-b8c4-3f1b01fa9a2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:16.198256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-9f03f36b-b476-4911-b8c4-3f1b01fa9a2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:16.200212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1353/2702] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:22:15 2022] DELETE /identity/v3/projects/fc733dcfcf9043feb354598fc2792ff5 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:22:16.208117 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea7ad1-6b35-4eec-b099-3c0668eee1e4 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:16.209119 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea7ad1-6b35-4eec-b099-3c0668eee1e4 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:16.209600 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea7ad1-6b35-4eec-b099-3c0668eee1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:16.242280 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-28ea7ad1-6b35-4eec-b099-3c0668eee1e4 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:22:16.243803 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-28ea7ad1-6b35-4eec-b099-3c0668eee1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:16.265584 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:16.334411 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1350/2703] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:22:16.343497 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a5557-4bec-4111-996d-5defedbd4a43 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:16.344598 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a5557-4bec-4111-996d-5defedbd4a43 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:16.345035 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a5557-4bec-4111-996d-5defedbd4a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:16.374506 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-cb7a5557-4bec-4111-996d-5defedbd4a43 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:22:16.376260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-cb7a5557-4bec-4111-996d-5defedbd4a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:16.389304 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1354/2704] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:22:16.394643 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea7ad1-6b35-4eec-b099-3c0668eee1e4 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:16.409977 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea7ad1-6b35-4eec-b099-3c0668eee1e4 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:16.416849 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f088da6-f999-456f-81fd-5ba52a394b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:16.418593 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f088da6-f999-456f-81fd-5ba52a394b2e admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:16.418593 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f088da6-f999-456f-81fd-5ba52a394b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:16.419640 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f088da6-f999-456f-81fd-5ba52a394b2e admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:16.420106 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f088da6-f999-456f-81fd-5ba52a394b2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:22:16.420690 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f088da6-f999-456f-81fd-5ba52a394b2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:16.423400 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f088da6-f999-456f-81fd-5ba52a394b2e admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:16.443787 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1351/2705] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:22:16.450659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a5557-4bec-4111-996d-5defedbd4a43 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:16.466799 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a5557-4bec-4111-996d-5defedbd4a43 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:16.474519 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36baaa-1f47-411e-ae9b-4d73b2feb887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:16.475160 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36baaa-1f47-411e-ae9b-4d73b2feb887 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:16.475522 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36baaa-1f47-411e-ae9b-4d73b2feb887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:16.475785 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36baaa-1f47-411e-ae9b-4d73b2feb887 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:16.476634 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36baaa-1f47-411e-ae9b-4d73b2feb887 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:16.479122 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36baaa-1f47-411e-ae9b-4d73b2feb887 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:16.497889 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:16.528266 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1355/2706] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:22:16 2022] POST /identity/v3/projects => generated 367 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:22:16.534953 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f088da6-f999-456f-81fd-5ba52a394b2e admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:16.551846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f088da6-f999-456f-81fd-5ba52a394b2e admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:16.635119 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4b91f-bad1-4308-a3ca-4e93da71ff2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:16.635633 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4b91f-bad1-4308-a3ca-4e93da71ff2d admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:16.635915 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4b91f-bad1-4308-a3ca-4e93da71ff2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:16.636192 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4b91f-bad1-4308-a3ca-4e93da71ff2d admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:16.636847 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4b91f-bad1-4308-a3ca-4e93da71ff2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:16.638813 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4b91f-bad1-4308-a3ca-4e93da71ff2d admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:16.667162 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1352/2707] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:22:16 2022] POST /identity/v3/users => generated 284 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:22:16.672746 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36baaa-1f47-411e-ae9b-4d73b2feb887 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:16.684924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36baaa-1f47-411e-ae9b-4d73b2feb887 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:16.690385 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055eba41-59d3-4287-88fd-20dadb0c1cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:16.691077 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055eba41-59d3-4287-88fd-20dadb0c1cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:16.691400 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055eba41-59d3-4287-88fd-20dadb0c1cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:16.691711 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055eba41-59d3-4287-88fd-20dadb0c1cf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:16.693006 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055eba41-59d3-4287-88fd-20dadb0c1cf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:16.694890 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055eba41-59d3-4287-88fd-20dadb0c1cf7 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:16.707468 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1356/2708] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:22:16.713480 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4b91f-bad1-4308-a3ca-4e93da71ff2d admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:16.728879 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4b91f-bad1-4308-a3ca-4e93da71ff2d admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:16.736393 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878405e-5c56-4533-8486-d16d3933bd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:16.737434 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878405e-5c56-4533-8486-d16d3933bd91 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:16.737970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878405e-5c56-4533-8486-d16d3933bd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:16.738462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878405e-5c56-4533-8486-d16d3933bd91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:16.740122 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878405e-5c56-4533-8486-d16d3933bd91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:16.741818 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878405e-5c56-4533-8486-d16d3933bd91 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:16.761100 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1353/2709] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:22:16.765835 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055eba41-59d3-4287-88fd-20dadb0c1cf7 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:16.777270 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055eba41-59d3-4287-88fd-20dadb0c1cf7 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:16.782666 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23275c90-78d1-4a2a-a71f-43e459729bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:16.783470 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23275c90-78d1-4a2a-a71f-43e459729bc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:16.783846 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23275c90-78d1-4a2a-a71f-43e459729bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:16.784218 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23275c90-78d1-4a2a-a71f-43e459729bc0 admin admin] PATH_INFO: `/v3/projects/f662b03666004755a1f7d481ce71e7f6/users/77a9452a8cf04ef39b9dc79730dab308/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:16.808737 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23275c90-78d1-4a2a-a71f-43e459729bc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f662b03666004755a1f7d481ce71e7f6, user_id=77a9452a8cf04ef39b9dc79730dab308, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:16.811932 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23275c90-78d1-4a2a-a71f-43e459729bc0 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:16.827329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1357/2710] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:22:16 2022] PUT /identity/v3/projects/f662b03666004755a1f7d481ce71e7f6/users/77a9452a8cf04ef39b9dc79730dab308/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:22:16.840582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14632b71-b4da-4b20-8db6-a8df0713a5b7 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:16.841003 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14632b71-b4da-4b20-8db6-a8df0713a5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:16.841421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14632b71-b4da-4b20-8db6-a8df0713a5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:16.877985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-14632b71-b4da-4b20-8db6-a8df0713a5b7 None None] MFA Rules not processed for user `77a9452a8cf04ef39b9dc79730dab308`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:22:16.880427 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-14632b71-b4da-4b20-8db6-a8df0713a5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:16.965771 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1354/2711] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:22:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:22:16.982180 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23275c90-78d1-4a2a-a71f-43e459729bc0 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:16.984837 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-23275c90-78d1-4a2a-a71f-43e459729bc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:17.078085 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23275c90-78d1-4a2a-a71f-43e459729bc0 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:17.082671 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c6b17-19ee-4726-862f-8b2f6fb0010a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:17.083743 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c6b17-19ee-4726-862f-8b2f6fb0010a service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:17.084232 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c6b17-19ee-4726-862f-8b2f6fb0010a service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:17.084703 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c6b17-19ee-4726-862f-8b2f6fb0010a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:17.098938 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c6b17-19ee-4726-862f-8b2f6fb0010a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:17.100858 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c6b17-19ee-4726-862f-8b2f6fb0010a service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:17.119308 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1358/2712] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:22:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:22:26.935157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14632b71-b4da-4b20-8db6-a8df0713a5b7 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:26.963192 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:27.036274 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14632b71-b4da-4b20-8db6-a8df0713a5b7 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:27.043964 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4653cc-f4d5-4a38-bf4c-0eb2b9279a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:27.044960 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4653cc-f4d5-4a38-bf4c-0eb2b9279a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:27.045409 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4653cc-f4d5-4a38-bf4c-0eb2b9279a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:27.045912 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4653cc-f4d5-4a38-bf4c-0eb2b9279a50 admin admin] PATH_INFO: `/v3/users/77a9452a8cf04ef39b9dc79730dab308` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:27.048147 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4653cc-f4d5-4a38-bf4c-0eb2b9279a50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77a9452a8cf04ef39b9dc79730dab308)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:27.052447 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4653cc-f4d5-4a38-bf4c-0eb2b9279a50 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:27.107887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:27.129127 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-7b4653cc-f4d5-4a38-bf4c-0eb2b9279a50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77a9452a8cf04ef39b9dc79730dab308'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:27.143684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-7b4653cc-f4d5-4a38-bf4c-0eb2b9279a50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77a9452a8cf04ef39b9dc79730dab308'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:27.165492 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-7b4653cc-f4d5-4a38-bf4c-0eb2b9279a50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77a9452a8cf04ef39b9dc79730dab308'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:27.176849 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-7b4653cc-f4d5-4a38-bf4c-0eb2b9279a50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77a9452a8cf04ef39b9dc79730dab308'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:27.177815 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-7b4653cc-f4d5-4a38-bf4c-0eb2b9279a50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77a9452a8cf04ef39b9dc79730dab308'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:27.186002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1355/2713] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:22:26 2022] DELETE /identity/v3/users/77a9452a8cf04ef39b9dc79730dab308 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:22:27.193748 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df589d5e-35b9-4ae4-9171-4d35af6588a5 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:27.194240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df589d5e-35b9-4ae4-9171-4d35af6588a5 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:27.194718 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df589d5e-35b9-4ae4-9171-4d35af6588a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:27.220705 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-df589d5e-35b9-4ae4-9171-4d35af6588a5 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:22:27.222237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-df589d5e-35b9-4ae4-9171-4d35af6588a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:27.327341 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1359/2714] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:22:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:22:27.339042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4653cc-f4d5-4a38-bf4c-0eb2b9279a50 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:27.341205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-7b4653cc-f4d5-4a38-bf4c-0eb2b9279a50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:27.435514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4653cc-f4d5-4a38-bf4c-0eb2b9279a50 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:27.439636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795182c9-bb5a-4ffa-8826-b7cdb827e3eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:27.441549 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795182c9-bb5a-4ffa-8826-b7cdb827e3eb service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:27.442005 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795182c9-bb5a-4ffa-8826-b7cdb827e3eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:27.442287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795182c9-bb5a-4ffa-8826-b7cdb827e3eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:27.455645 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795182c9-bb5a-4ffa-8826-b7cdb827e3eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:27.458202 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795182c9-bb5a-4ffa-8826-b7cdb827e3eb service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:27.474087 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1356/2715] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:22:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:22:27.906847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df589d5e-35b9-4ae4-9171-4d35af6588a5 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:27.909351 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-df589d5e-35b9-4ae4-9171-4d35af6588a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:27.931776 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df589d5e-35b9-4ae4-9171-4d35af6588a5 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:27.937218 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8004e-923c-4c11-857a-b318ef9f06fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:27.938098 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8004e-923c-4c11-857a-b318ef9f06fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:27.938290 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8004e-923c-4c11-857a-b318ef9f06fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:27.938790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8004e-923c-4c11-857a-b318ef9f06fc admin admin] PATH_INFO: `/v3/projects/f662b03666004755a1f7d481ce71e7f6` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:27.942294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8004e-923c-4c11-857a-b318ef9f06fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f662b03666004755a1f7d481ce71e7f6)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:27.944359 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8004e-923c-4c11-857a-b318ef9f06fc admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:28.030580 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d9c8004e-923c-4c11-857a-b318ef9f06fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f662b03666004755a1f7d481ce71e7f6', 'request_id': 'req-d9c8004e-923c-4c11-857a-b318ef9f06fc'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:28.039905 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d9c8004e-923c-4c11-857a-b318ef9f06fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f662b03666004755a1f7d481ce71e7f6', 'request_id': 'req-d9c8004e-923c-4c11-857a-b318ef9f06fc'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:28.052845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d9c8004e-923c-4c11-857a-b318ef9f06fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f662b03666004755a1f7d481ce71e7f6', 'request_id': 'req-d9c8004e-923c-4c11-857a-b318ef9f06fc'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:28.071600 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d9c8004e-923c-4c11-857a-b318ef9f06fc admin admin] The token cache is being invalidate because project f662b03666004755a1f7d481ce71e7f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:22:28.071986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d9c8004e-923c-4c11-857a-b318ef9f06fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:28.072865 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d9c8004e-923c-4c11-857a-b318ef9f06fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:28.074632 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1360/2716] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:22:27 2022] DELETE /identity/v3/projects/f662b03666004755a1f7d481ce71e7f6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:22:28.080120 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad829f43-018e-47fc-9a72-7525146ae6a0 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:28.080887 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad829f43-018e-47fc-9a72-7525146ae6a0 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:28.080887 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad829f43-018e-47fc-9a72-7525146ae6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:28.113226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-ad829f43-018e-47fc-9a72-7525146ae6a0 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:22:28.116020 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-ad829f43-018e-47fc-9a72-7525146ae6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:28.177902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:28.232271 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1357/2717] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:22:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:22:28.245270 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8332e-7c8b-494a-97ab-cd0d991a0cd1 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:28.246049 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8332e-7c8b-494a-97ab-cd0d991a0cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:28.246049 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8332e-7c8b-494a-97ab-cd0d991a0cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:28.273052 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-32a8332e-7c8b-494a-97ab-cd0d991a0cd1 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:22:28.274353 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-32a8332e-7c8b-494a-97ab-cd0d991a0cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:28.284920 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1361/2718] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:22:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:22:28.289542 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad829f43-018e-47fc-9a72-7525146ae6a0 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:28.301258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad829f43-018e-47fc-9a72-7525146ae6a0 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:28.306389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8665ea-ec51-427d-b6de-b62a3e2dc92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:28.307166 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8665ea-ec51-427d-b6de-b62a3e2dc92c admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:28.307603 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8665ea-ec51-427d-b6de-b62a3e2dc92c admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:28.308019 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8665ea-ec51-427d-b6de-b62a3e2dc92c admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:28.308905 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8665ea-ec51-427d-b6de-b62a3e2dc92c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:22:28.309306 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8665ea-ec51-427d-b6de-b62a3e2dc92c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:28.311054 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8665ea-ec51-427d-b6de-b62a3e2dc92c admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:28.325459 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1358/2719] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:22:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:22:28.330165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a8332e-7c8b-494a-97ab-cd0d991a0cd1 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:28.340809 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a8332e-7c8b-494a-97ab-cd0d991a0cd1 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:28.346551 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ed76d-5cdf-454a-8555-42e4bde6001b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:28.347199 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ed76d-5cdf-454a-8555-42e4bde6001b admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:28.347515 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ed76d-5cdf-454a-8555-42e4bde6001b admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:28.347835 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ed76d-5cdf-454a-8555-42e4bde6001b admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:28.348583 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ed76d-5cdf-454a-8555-42e4bde6001b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:28.350763 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ed76d-5cdf-454a-8555-42e4bde6001b admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:28.419214 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1362/2720] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:22:28 2022] POST /identity/v3/projects => generated 405 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:22:28.425833 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8665ea-ec51-427d-b6de-b62a3e2dc92c admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:28.443680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8665ea-ec51-427d-b6de-b62a3e2dc92c admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:28.561560 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8b10f-575e-43d8-91fd-f9b6b9aba051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:28.562589 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8b10f-575e-43d8-91fd-f9b6b9aba051 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:28.563071 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8b10f-575e-43d8-91fd-f9b6b9aba051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:28.563574 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8b10f-575e-43d8-91fd-f9b6b9aba051 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:28.565289 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8b10f-575e-43d8-91fd-f9b6b9aba051 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:28.569120 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8b10f-575e-43d8-91fd-f9b6b9aba051 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:28.603422 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1359/2721] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:22:28 2022] POST /identity/v3/users => generated 303 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:22:28.608368 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ed76d-5cdf-454a-8555-42e4bde6001b admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:28.625943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ed76d-5cdf-454a-8555-42e4bde6001b admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:28.633478 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a356310f-2290-4d93-a07a-654d3e09f567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:28.634377 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356310f-2290-4d93-a07a-654d3e09f567 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:28.634797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356310f-2290-4d93-a07a-654d3e09f567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:28.635202 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356310f-2290-4d93-a07a-654d3e09f567 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:28.636175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a356310f-2290-4d93-a07a-654d3e09f567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:28.638837 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a356310f-2290-4d93-a07a-654d3e09f567 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:28.657895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1363/2722] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:22:28.664193 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8b10f-575e-43d8-91fd-f9b6b9aba051 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:28.679111 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8b10f-575e-43d8-91fd-f9b6b9aba051 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:28.684114 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1de2d1-a234-49db-b55a-09191903d314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:28.684844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1de2d1-a234-49db-b55a-09191903d314 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:28.685169 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1de2d1-a234-49db-b55a-09191903d314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:28.685548 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1de2d1-a234-49db-b55a-09191903d314 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:28.686371 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1de2d1-a234-49db-b55a-09191903d314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:28.688467 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1de2d1-a234-49db-b55a-09191903d314 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:28.712995 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1360/2723] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:22:28.717058 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a356310f-2290-4d93-a07a-654d3e09f567 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:28.729655 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a356310f-2290-4d93-a07a-654d3e09f567 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:28.737060 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ecf08-d673-4731-8d77-7c127eff61ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:28.737876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ecf08-d673-4731-8d77-7c127eff61ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:28.738314 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ecf08-d673-4731-8d77-7c127eff61ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:28.738723 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ecf08-d673-4731-8d77-7c127eff61ca admin admin] PATH_INFO: `/v3/projects/ba6a26afe1114b189471efd02c46b633/users/97189d8526344e358663f827ee707066/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:28.760758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ecf08-d673-4731-8d77-7c127eff61ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba6a26afe1114b189471efd02c46b633, user_id=97189d8526344e358663f827ee707066, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:28.765266 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ecf08-d673-4731-8d77-7c127eff61ca admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:28.787420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1364/2724] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:22:28 2022] PUT /identity/v3/projects/ba6a26afe1114b189471efd02c46b633/users/97189d8526344e358663f827ee707066/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:22:28.794811 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956cba2-fec9-40c0-b369-464a02440951 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:28.795287 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956cba2-fec9-40c0-b369-464a02440951 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:28.795697 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956cba2-fec9-40c0-b369-464a02440951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:28.826877 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-9956cba2-fec9-40c0-b369-464a02440951 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:22:28.828572 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9956cba2-fec9-40c0-b369-464a02440951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:28.953668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1361/2725] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:22:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:22:28.966986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ecf08-d673-4731-8d77-7c127eff61ca admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:28.969801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-113ecf08-d673-4731-8d77-7c127eff61ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:28.994710 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:29.085511 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ecf08-d673-4731-8d77-7c127eff61ca admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:29.091831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9111a86-a626-413d-9ff4-96293054b1d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:29.092527 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9111a86-a626-413d-9ff4-96293054b1d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:29.092960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9111a86-a626-413d-9ff4-96293054b1d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:29.094004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9111a86-a626-413d-9ff4-96293054b1d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:29.108076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9111a86-a626-413d-9ff4-96293054b1d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:29.109747 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9111a86-a626-413d-9ff4-96293054b1d9 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:29.123652 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1365/2726] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:22:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:22:30.117023 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b1478-d431-4ffa-83d3-85487940ad64 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:30.117881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b1478-d431-4ffa-83d3-85487940ad64 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:30.118051 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b1478-d431-4ffa-83d3-85487940ad64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:30.166986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-177b1478-d431-4ffa-83d3-85487940ad64 None None] MFA Rules not processed for user `97189d8526344e358663f827ee707066`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:22:30.168689 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-177b1478-d431-4ffa-83d3-85487940ad64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:30.231103 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:30.303908 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1362/2727] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:22:30 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:22:30.309226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9111a86-a626-413d-9ff4-96293054b1d9 service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:30.326829 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9111a86-a626-413d-9ff4-96293054b1d9 service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:30.340143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6196de-95b9-46b0-816c-430d6e764a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:30.341779 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6196de-95b9-46b0-816c-430d6e764a83 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:30.341779 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6196de-95b9-46b0-816c-430d6e764a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:30.342342 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6196de-95b9-46b0-816c-430d6e764a83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:30.343062 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6196de-95b9-46b0-816c-430d6e764a83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:30.346634 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6196de-95b9-46b0-816c-430d6e764a83 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:30.392935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1366/2728] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:22:30 2022] POST /identity/v3/projects => generated 403 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:22:30.399540 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b1478-d431-4ffa-83d3-85487940ad64 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:30.422460 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b1478-d431-4ffa-83d3-85487940ad64 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:30.516654 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8541d2-5417-4fc2-b6cf-d53ce76d1cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:30.518232 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8541d2-5417-4fc2-b6cf-d53ce76d1cb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:30.518729 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8541d2-5417-4fc2-b6cf-d53ce76d1cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:30.519223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8541d2-5417-4fc2-b6cf-d53ce76d1cb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:30.520293 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8541d2-5417-4fc2-b6cf-d53ce76d1cb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:30.523685 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8541d2-5417-4fc2-b6cf-d53ce76d1cb8 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:30.559636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1363/2729] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:22:30 2022] POST /identity/v3/users => generated 308 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:22:30.568614 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6196de-95b9-46b0-816c-430d6e764a83 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:30.586564 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6196de-95b9-46b0-816c-430d6e764a83 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:30.594246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0943fe-1115-46ac-aeba-3a55ad1e913e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:30.595284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0943fe-1115-46ac-aeba-3a55ad1e913e admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:30.596795 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0943fe-1115-46ac-aeba-3a55ad1e913e admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:30.597381 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0943fe-1115-46ac-aeba-3a55ad1e913e admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:30.598820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0943fe-1115-46ac-aeba-3a55ad1e913e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:30.601698 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0943fe-1115-46ac-aeba-3a55ad1e913e admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:30.622704 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1367/2730] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:22:30.624291 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8541d2-5417-4fc2-b6cf-d53ce76d1cb8 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:30.642183 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8541d2-5417-4fc2-b6cf-d53ce76d1cb8 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:30.648841 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75a6ff-1366-4539-93d2-1cf77d32f525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:30.650011 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75a6ff-1366-4539-93d2-1cf77d32f525 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:30.650387 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75a6ff-1366-4539-93d2-1cf77d32f525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:30.650809 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75a6ff-1366-4539-93d2-1cf77d32f525 admin admin] PATH_INFO: `/v3/projects/88dd75f0b70b47dab83d8fe89bca4a55/users/837b952544c244d884da3e61f9a86868/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:30.677191 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e75a6ff-1366-4539-93d2-1cf77d32f525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88dd75f0b70b47dab83d8fe89bca4a55, user_id=837b952544c244d884da3e61f9a86868, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:30.681091 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e75a6ff-1366-4539-93d2-1cf77d32f525 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:30.701599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1364/2731] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:22:30 2022] PUT /identity/v3/projects/88dd75f0b70b47dab83d8fe89bca4a55/users/837b952544c244d884da3e61f9a86868/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:22:30.711304 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0943fe-1115-46ac-aeba-3a55ad1e913e admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:30.723627 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0943fe-1115-46ac-aeba-3a55ad1e913e admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:30.808863 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca79f77-060d-4d1e-b3af-128778ba082c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:30.809807 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca79f77-060d-4d1e-b3af-128778ba082c admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:30.810489 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca79f77-060d-4d1e-b3af-128778ba082c admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:30.811109 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca79f77-060d-4d1e-b3af-128778ba082c admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:30.812874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca79f77-060d-4d1e-b3af-128778ba082c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:30.815377 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca79f77-060d-4d1e-b3af-128778ba082c admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:30.834758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1368/2732] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:22:30.841282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75a6ff-1366-4539-93d2-1cf77d32f525 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:30.857534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75a6ff-1366-4539-93d2-1cf77d32f525 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:30.867107 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140cfac1-6b5d-4f2c-a1a5-d1707f86c51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:30.867858 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cfac1-6b5d-4f2c-a1a5-d1707f86c51e admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:30.868294 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cfac1-6b5d-4f2c-a1a5-d1707f86c51e admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:30.868734 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cfac1-6b5d-4f2c-a1a5-d1707f86c51e admin admin] PATH_INFO: `/v3/projects/88dd75f0b70b47dab83d8fe89bca4a55/users/837b952544c244d884da3e61f9a86868/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:30.871567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140cfac1-6b5d-4f2c-a1a5-d1707f86c51e admin admin] RBAC: Authorizing `identity:create_grant(project_id=88dd75f0b70b47dab83d8fe89bca4a55, user_id=837b952544c244d884da3e61f9a86868, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:30.875590 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140cfac1-6b5d-4f2c-a1a5-d1707f86c51e admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:30.896032 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1365/2733] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:22:30 2022] PUT /identity/v3/projects/88dd75f0b70b47dab83d8fe89bca4a55/users/837b952544c244d884da3e61f9a86868/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:22:31.948429 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798af2fd-e6d8-4050-a51f-406c0745da36 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:31.949318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798af2fd-e6d8-4050-a51f-406c0745da36 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:31.949531 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798af2fd-e6d8-4050-a51f-406c0745da36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:32.010947 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-798af2fd-e6d8-4050-a51f-406c0745da36 None None] MFA Rules not processed for user `837b952544c244d884da3e61f9a86868`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:22:32.016929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-798af2fd-e6d8-4050-a51f-406c0745da36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:32.158401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1369/2734] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:22:31 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:22:33.741534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140cfac1-6b5d-4f2c-a1a5-d1707f86c51e admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:33.861904 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140cfac1-6b5d-4f2c-a1a5-d1707f86c51e admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:33.874021 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b89a5-5cca-4484-bf22-107cd666c714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:33.875218 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b89a5-5cca-4484-bf22-107cd666c714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:33.875727 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b89a5-5cca-4484-bf22-107cd666c714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:33.876829 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b89a5-5cca-4484-bf22-107cd666c714 admin admin] PATH_INFO: `/v3/users/97189d8526344e358663f827ee707066` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:33.879214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b89a5-5cca-4484-bf22-107cd666c714 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97189d8526344e358663f827ee707066)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:33.883489 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b89a5-5cca-4484-bf22-107cd666c714 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:33.939927 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:33.963549 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-128b89a5-5cca-4484-bf22-107cd666c714 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97189d8526344e358663f827ee707066'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:33.974521 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-128b89a5-5cca-4484-bf22-107cd666c714 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97189d8526344e358663f827ee707066'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:33.993310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-128b89a5-5cca-4484-bf22-107cd666c714 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97189d8526344e358663f827ee707066'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:34.005136 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-128b89a5-5cca-4484-bf22-107cd666c714 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97189d8526344e358663f827ee707066'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:34.006610 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-128b89a5-5cca-4484-bf22-107cd666c714 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97189d8526344e358663f827ee707066'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:34.018674 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1366/2735] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:22:33 2022] DELETE /identity/v3/users/97189d8526344e358663f827ee707066 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:22:34.293114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798af2fd-e6d8-4050-a51f-406c0745da36 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:34.302389 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:34.311945 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798af2fd-e6d8-4050-a51f-406c0745da36 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:34.405812 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f3e0bd-436a-4ea8-a14d-dd6e1e57849b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:34.406733 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3e0bd-436a-4ea8-a14d-dd6e1e57849b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:34.407258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3e0bd-436a-4ea8-a14d-dd6e1e57849b admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:34.407640 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3e0bd-436a-4ea8-a14d-dd6e1e57849b admin admin] PATH_INFO: `/v3/projects/ba6a26afe1114b189471efd02c46b633` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:34.409586 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f3e0bd-436a-4ea8-a14d-dd6e1e57849b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba6a26afe1114b189471efd02c46b633)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:34.412592 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f3e0bd-436a-4ea8-a14d-dd6e1e57849b admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:34.479331 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d4f3e0bd-436a-4ea8-a14d-dd6e1e57849b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba6a26afe1114b189471efd02c46b633', 'request_id': 'req-d4f3e0bd-436a-4ea8-a14d-dd6e1e57849b'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:34.488159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d4f3e0bd-436a-4ea8-a14d-dd6e1e57849b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba6a26afe1114b189471efd02c46b633', 'request_id': 'req-d4f3e0bd-436a-4ea8-a14d-dd6e1e57849b'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:34.498558 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d4f3e0bd-436a-4ea8-a14d-dd6e1e57849b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba6a26afe1114b189471efd02c46b633', 'request_id': 'req-d4f3e0bd-436a-4ea8-a14d-dd6e1e57849b'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:34.520433 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d4f3e0bd-436a-4ea8-a14d-dd6e1e57849b admin admin] The token cache is being invalidate because project ba6a26afe1114b189471efd02c46b633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:22:34.520869 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d4f3e0bd-436a-4ea8-a14d-dd6e1e57849b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:34.521876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-d4f3e0bd-436a-4ea8-a14d-dd6e1e57849b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:34.524150 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1370/2736] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:22:34 2022] DELETE /identity/v3/projects/ba6a26afe1114b189471efd02c46b633 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:22:34.533310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b89a5-5cca-4484-bf22-107cd666c714 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:34.535340 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-128b89a5-5cca-4484-bf22-107cd666c714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:34.555401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b89a5-5cca-4484-bf22-107cd666c714 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:34.647458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b9d14a-e8e5-4c13-a199-19d390cdb83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:34.648146 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b9d14a-e8e5-4c13-a199-19d390cdb83e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:34.648606 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b9d14a-e8e5-4c13-a199-19d390cdb83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:34.649058 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b9d14a-e8e5-4c13-a199-19d390cdb83e admin admin] PATH_INFO: `/v3/users/837b952544c244d884da3e61f9a86868` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:34.651059 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b9d14a-e8e5-4c13-a199-19d390cdb83e admin admin] RBAC: Authorizing `identity:delete_user(user_id=837b952544c244d884da3e61f9a86868)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:34.653183 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b9d14a-e8e5-4c13-a199-19d390cdb83e admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:34.726763 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b2b9d14a-e8e5-4c13-a199-19d390cdb83e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '837b952544c244d884da3e61f9a86868'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:34.744564 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b2b9d14a-e8e5-4c13-a199-19d390cdb83e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '837b952544c244d884da3e61f9a86868'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:34.766221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b2b9d14a-e8e5-4c13-a199-19d390cdb83e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '837b952544c244d884da3e61f9a86868'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:34.778844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b2b9d14a-e8e5-4c13-a199-19d390cdb83e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '837b952544c244d884da3e61f9a86868'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:34.779893 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b2b9d14a-e8e5-4c13-a199-19d390cdb83e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '837b952544c244d884da3e61f9a86868'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:34.790573 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1367/2737] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:22:34 2022] DELETE /identity/v3/users/837b952544c244d884da3e61f9a86868 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:22:35.083482 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f3e0bd-436a-4ea8-a14d-dd6e1e57849b admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:35.086032 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-d4f3e0bd-436a-4ea8-a14d-dd6e1e57849b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:35.116950 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f3e0bd-436a-4ea8-a14d-dd6e1e57849b admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:35.215295 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b291ae63-1e6d-433d-b6fb-7dc964ff83c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:35.216140 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b291ae63-1e6d-433d-b6fb-7dc964ff83c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:35.216759 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b291ae63-1e6d-433d-b6fb-7dc964ff83c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:35.217177 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b291ae63-1e6d-433d-b6fb-7dc964ff83c0 admin admin] PATH_INFO: `/v3/projects/88dd75f0b70b47dab83d8fe89bca4a55` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:35.218952 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b291ae63-1e6d-433d-b6fb-7dc964ff83c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88dd75f0b70b47dab83d8fe89bca4a55)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:35.223001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b291ae63-1e6d-433d-b6fb-7dc964ff83c0 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:35.280877 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:35.300845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-b291ae63-1e6d-433d-b6fb-7dc964ff83c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88dd75f0b70b47dab83d8fe89bca4a55', 'request_id': 'req-b291ae63-1e6d-433d-b6fb-7dc964ff83c0'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:35.310053 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-b291ae63-1e6d-433d-b6fb-7dc964ff83c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88dd75f0b70b47dab83d8fe89bca4a55', 'request_id': 'req-b291ae63-1e6d-433d-b6fb-7dc964ff83c0'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:35.321008 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-b291ae63-1e6d-433d-b6fb-7dc964ff83c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88dd75f0b70b47dab83d8fe89bca4a55', 'request_id': 'req-b291ae63-1e6d-433d-b6fb-7dc964ff83c0'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:35.340167 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-b291ae63-1e6d-433d-b6fb-7dc964ff83c0 admin admin] The token cache is being invalidate because project 88dd75f0b70b47dab83d8fe89bca4a55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:22:35.340528 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-b291ae63-1e6d-433d-b6fb-7dc964ff83c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:35.341388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-b291ae63-1e6d-433d-b6fb-7dc964ff83c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:35.343489 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1371/2738] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:22:35 2022] DELETE /identity/v3/projects/88dd75f0b70b47dab83d8fe89bca4a55 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:22:35.353252 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111c115-03a0-4a90-9cfa-87f35dd520e6 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:35.353734 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111c115-03a0-4a90-9cfa-87f35dd520e6 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:35.354193 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111c115-03a0-4a90-9cfa-87f35dd520e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:35.391257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-e111c115-03a0-4a90-9cfa-87f35dd520e6 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:22:35.393497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-e111c115-03a0-4a90-9cfa-87f35dd520e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:35.428639 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:35.493332 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1368/2739] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:22:35.504907 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ef689-97db-4f02-9863-5674b1e7a7fd None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:35.505219 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ef689-97db-4f02-9863-5674b1e7a7fd None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:35.505763 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ef689-97db-4f02-9863-5674b1e7a7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:35.537935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-196ef689-97db-4f02-9863-5674b1e7a7fd None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:22:35.539860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-196ef689-97db-4f02-9863-5674b1e7a7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:35.553303 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1372/2740] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:22:35.560168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111c115-03a0-4a90-9cfa-87f35dd520e6 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:35.572546 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111c115-03a0-4a90-9cfa-87f35dd520e6 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:35.585941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6427e68-fefc-410e-9828-302d7ef84bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:35.586705 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6427e68-fefc-410e-9828-302d7ef84bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:35.587091 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6427e68-fefc-410e-9828-302d7ef84bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:35.587423 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6427e68-fefc-410e-9828-302d7ef84bf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:35.588240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6427e68-fefc-410e-9828-302d7ef84bf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:22:35.588554 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6427e68-fefc-410e-9828-302d7ef84bf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:35.591036 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6427e68-fefc-410e-9828-302d7ef84bf9 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:35.611488 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1369/2741] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:22:35.618904 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ef689-97db-4f02-9863-5674b1e7a7fd None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:35.633529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ef689-97db-4f02-9863-5674b1e7a7fd None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:35.643046 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb50a55-5df5-4abc-b6cb-aa9713597e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:35.644029 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb50a55-5df5-4abc-b6cb-aa9713597e7f admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:35.644339 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb50a55-5df5-4abc-b6cb-aa9713597e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:35.644623 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb50a55-5df5-4abc-b6cb-aa9713597e7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:35.645475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb50a55-5df5-4abc-b6cb-aa9713597e7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:35.648447 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb50a55-5df5-4abc-b6cb-aa9713597e7f admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:35.710823 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1373/2742] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:22:35 2022] POST /identity/v3/projects => generated 387 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:22:35.715736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6427e68-fefc-410e-9828-302d7ef84bf9 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:35.727691 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6427e68-fefc-410e-9828-302d7ef84bf9 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:35.798333 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092f667-481c-47e9-8d35-ddbca7581777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:35.799315 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092f667-481c-47e9-8d35-ddbca7581777 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:35.799315 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092f667-481c-47e9-8d35-ddbca7581777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:35.799680 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092f667-481c-47e9-8d35-ddbca7581777 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:35.800165 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092f667-481c-47e9-8d35-ddbca7581777 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:35.802199 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092f667-481c-47e9-8d35-ddbca7581777 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:35.838000 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1370/2743] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:22:35 2022] POST /identity/v3/users => generated 294 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:22:35.844047 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb50a55-5df5-4abc-b6cb-aa9713597e7f admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:35.861062 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb50a55-5df5-4abc-b6cb-aa9713597e7f admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:35.868468 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75b399-0c6e-48fd-8d56-05e82316fbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:35.869190 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75b399-0c6e-48fd-8d56-05e82316fbb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:35.869509 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75b399-0c6e-48fd-8d56-05e82316fbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:35.869851 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75b399-0c6e-48fd-8d56-05e82316fbb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:35.870568 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75b399-0c6e-48fd-8d56-05e82316fbb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:35.872796 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75b399-0c6e-48fd-8d56-05e82316fbb7 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:35.889869 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1374/2744] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:22:35.895143 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092f667-481c-47e9-8d35-ddbca7581777 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:35.915522 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092f667-481c-47e9-8d35-ddbca7581777 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:35.924310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d4f04-240a-4117-96e0-31576a4956aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:35.925361 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d4f04-240a-4117-96e0-31576a4956aa admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:35.925865 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d4f04-240a-4117-96e0-31576a4956aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:35.926265 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d4f04-240a-4117-96e0-31576a4956aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:35.927296 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d4f04-240a-4117-96e0-31576a4956aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:35.930193 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d4f04-240a-4117-96e0-31576a4956aa admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:35.950919 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1371/2745] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:22:35.957143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75b399-0c6e-48fd-8d56-05e82316fbb7 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:35.974850 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75b399-0c6e-48fd-8d56-05e82316fbb7 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:35.982580 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ab426-4312-41f7-82e1-f9c6a157b77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:35.983757 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ab426-4312-41f7-82e1-f9c6a157b77c admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:35.984133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ab426-4312-41f7-82e1-f9c6a157b77c admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:35.984411 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ab426-4312-41f7-82e1-f9c6a157b77c admin admin] PATH_INFO: `/v3/projects/986d9add578a4688864fb9fcd016496d/users/5719dc470b9a4e9f86012195de100bbd/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:36.004955 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ab426-4312-41f7-82e1-f9c6a157b77c admin admin] RBAC: Authorizing `identity:create_grant(project_id=986d9add578a4688864fb9fcd016496d, user_id=5719dc470b9a4e9f86012195de100bbd, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:36.008402 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ab426-4312-41f7-82e1-f9c6a157b77c admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:36.029818 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1375/2746] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:22:35 2022] PUT /identity/v3/projects/986d9add578a4688864fb9fcd016496d/users/5719dc470b9a4e9f86012195de100bbd/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:22:36.043901 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390fd7d-d698-4de9-b215-a8c50ff2dc3b None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:36.044192 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390fd7d-d698-4de9-b215-a8c50ff2dc3b None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:36.044472 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390fd7d-d698-4de9-b215-a8c50ff2dc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:36.089603 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-4390fd7d-d698-4de9-b215-a8c50ff2dc3b None None] MFA Rules not processed for user `5719dc470b9a4e9f86012195de100bbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:22:36.091978 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-4390fd7d-d698-4de9-b215-a8c50ff2dc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:36.201953 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1372/2747] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:22:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:22:36.209001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ab426-4312-41f7-82e1-f9c6a157b77c admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:36.318411 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ab426-4312-41f7-82e1-f9c6a157b77c admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:36.323432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff445f4f-baa7-4030-a44d-af26d34774f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:36.323843 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff445f4f-baa7-4030-a44d-af26d34774f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:36.324184 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff445f4f-baa7-4030-a44d-af26d34774f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:36.324518 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff445f4f-baa7-4030-a44d-af26d34774f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:36.325236 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff445f4f-baa7-4030-a44d-af26d34774f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:36.327753 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff445f4f-baa7-4030-a44d-af26d34774f5 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:36.371168 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1376/2748] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:22:36 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:22:36.376190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390fd7d-d698-4de9-b215-a8c50ff2dc3b None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:36.393177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390fd7d-d698-4de9-b215-a8c50ff2dc3b None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:36.425621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:36.487540 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e3ec4-47ef-471c-ad93-72de5c99999a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:36.488182 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e3ec4-47ef-471c-ad93-72de5c99999a admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:36.488461 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e3ec4-47ef-471c-ad93-72de5c99999a admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:36.488767 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e3ec4-47ef-471c-ad93-72de5c99999a admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:36.489437 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e3ec4-47ef-471c-ad93-72de5c99999a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:36.491581 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e3ec4-47ef-471c-ad93-72de5c99999a admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:36.518382 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1373/2749] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:22:36 2022] POST /identity/v3/users => generated 299 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:22:36.522740 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff445f4f-baa7-4030-a44d-af26d34774f5 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:36.535093 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff445f4f-baa7-4030-a44d-af26d34774f5 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:36.540337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062b13e-673a-4fc5-86ce-ef131b38146a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:36.540966 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062b13e-673a-4fc5-86ce-ef131b38146a admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:36.541243 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062b13e-673a-4fc5-86ce-ef131b38146a admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:36.541508 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062b13e-673a-4fc5-86ce-ef131b38146a admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:36.542197 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e062b13e-673a-4fc5-86ce-ef131b38146a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:36.543814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e062b13e-673a-4fc5-86ce-ef131b38146a admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:36.556907 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1377/2750] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:22:36.563056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e3ec4-47ef-471c-ad93-72de5c99999a admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:36.581737 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e3ec4-47ef-471c-ad93-72de5c99999a admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:36.587144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceda6f1-e3b4-43ba-8b28-cbc768ece034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:36.587840 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceda6f1-e3b4-43ba-8b28-cbc768ece034 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:36.588116 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceda6f1-e3b4-43ba-8b28-cbc768ece034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:36.588383 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceda6f1-e3b4-43ba-8b28-cbc768ece034 admin admin] PATH_INFO: `/v3/projects/decd7571e1484329b02f3d407d746f46/users/33502be8431047db880c7842e51c7c9b/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:36.614613 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceda6f1-e3b4-43ba-8b28-cbc768ece034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=decd7571e1484329b02f3d407d746f46, user_id=33502be8431047db880c7842e51c7c9b, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:36.617317 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceda6f1-e3b4-43ba-8b28-cbc768ece034 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:36.637545 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1374/2751] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:22:36 2022] PUT /identity/v3/projects/decd7571e1484329b02f3d407d746f46/users/33502be8431047db880c7842e51c7c9b/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:22:36.644682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062b13e-673a-4fc5-86ce-ef131b38146a admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:36.666427 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062b13e-673a-4fc5-86ce-ef131b38146a admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:36.710258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:36.744937 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69b4eb-ee81-4258-ae2d-a8061d5de735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:36.745560 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69b4eb-ee81-4258-ae2d-a8061d5de735 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:36.745917 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69b4eb-ee81-4258-ae2d-a8061d5de735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:36.746252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69b4eb-ee81-4258-ae2d-a8061d5de735 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:36.747099 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e69b4eb-ee81-4258-ae2d-a8061d5de735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:36.748748 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e69b4eb-ee81-4258-ae2d-a8061d5de735 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:36.769277 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1378/2752] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:22:36.774409 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceda6f1-e3b4-43ba-8b28-cbc768ece034 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:36.791921 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceda6f1-e3b4-43ba-8b28-cbc768ece034 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:36.800045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebd203-1d03-42d9-9903-d9bd47d47ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:36.801117 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebd203-1d03-42d9-9903-d9bd47d47ab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:36.801485 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebd203-1d03-42d9-9903-d9bd47d47ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:36.801798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebd203-1d03-42d9-9903-d9bd47d47ab1 admin admin] PATH_INFO: `/v3/projects/decd7571e1484329b02f3d407d746f46/users/33502be8431047db880c7842e51c7c9b/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:36.804529 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ebd203-1d03-42d9-9903-d9bd47d47ab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=decd7571e1484329b02f3d407d746f46, user_id=33502be8431047db880c7842e51c7c9b, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:36.807634 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ebd203-1d03-42d9-9903-d9bd47d47ab1 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:36.822143 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1375/2753] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:22:36 2022] PUT /identity/v3/projects/decd7571e1484329b02f3d407d746f46/users/33502be8431047db880c7842e51c7c9b/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:22:36.839507 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196ee4c-80a3-4776-abf3-23cff40803d5 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:36.839900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196ee4c-80a3-4776-abf3-23cff40803d5 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:36.840373 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196ee4c-80a3-4776-abf3-23cff40803d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:36.881137 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-3196ee4c-80a3-4776-abf3-23cff40803d5 None None] MFA Rules not processed for user `33502be8431047db880c7842e51c7c9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:22:36.883029 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-3196ee4c-80a3-4776-abf3-23cff40803d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:36.982196 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1379/2754] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:22:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:22:37.000315 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebd203-1d03-42d9-9903-d9bd47d47ab1 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:37.004114 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-93ebd203-1d03-42d9-9903-d9bd47d47ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:37.125038 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebd203-1d03-42d9-9903-d9bd47d47ab1 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:37.132334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382976c-0761-4fcb-a0fd-3ab3c2fc91a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:37.133191 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382976c-0761-4fcb-a0fd-3ab3c2fc91a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:37.133572 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382976c-0761-4fcb-a0fd-3ab3c2fc91a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:37.133971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382976c-0761-4fcb-a0fd-3ab3c2fc91a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:37.148779 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382976c-0761-4fcb-a0fd-3ab3c2fc91a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:37.150842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382976c-0761-4fcb-a0fd-3ab3c2fc91a5 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:37.165895 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1376/2755] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:22:36 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:22:37.263345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196ee4c-80a3-4776-abf3-23cff40803d5 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:37.363683 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196ee4c-80a3-4776-abf3-23cff40803d5 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:37.371223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec82030-284d-499b-8418-43009bab1d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:37.372185 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec82030-284d-499b-8418-43009bab1d07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:37.372644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec82030-284d-499b-8418-43009bab1d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:37.373167 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec82030-284d-499b-8418-43009bab1d07 admin admin] PATH_INFO: `/v3/users/5719dc470b9a4e9f86012195de100bbd` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:37.375077 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec82030-284d-499b-8418-43009bab1d07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5719dc470b9a4e9f86012195de100bbd)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:37.378299 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec82030-284d-499b-8418-43009bab1d07 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:37.456486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-aec82030-284d-499b-8418-43009bab1d07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5719dc470b9a4e9f86012195de100bbd'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:37.469860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:37.479153 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-aec82030-284d-499b-8418-43009bab1d07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5719dc470b9a4e9f86012195de100bbd'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:37.493600 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-aec82030-284d-499b-8418-43009bab1d07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5719dc470b9a4e9f86012195de100bbd'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:37.504005 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-aec82030-284d-499b-8418-43009bab1d07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5719dc470b9a4e9f86012195de100bbd'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:37.504802 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-aec82030-284d-499b-8418-43009bab1d07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5719dc470b9a4e9f86012195de100bbd'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:37.511380 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1380/2756] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:22:37 2022] DELETE /identity/v3/users/5719dc470b9a4e9f86012195de100bbd => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:22:37.516886 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c8123-91cb-4c8e-9ebf-64007247801c None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:37.517290 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c8123-91cb-4c8e-9ebf-64007247801c None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:37.517719 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c8123-91cb-4c8e-9ebf-64007247801c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:37.538700 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-f23c8123-91cb-4c8e-9ebf-64007247801c None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:22:37.539949 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-f23c8123-91cb-4c8e-9ebf-64007247801c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:37.634586 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1377/2757] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:22:37.647168 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec82030-284d-499b-8418-43009bab1d07 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:37.648614 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-aec82030-284d-499b-8418-43009bab1d07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:37.736537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec82030-284d-499b-8418-43009bab1d07 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:37.737631 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d3507-6025-436d-ae55-06be5ff09eb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:37.738289 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d3507-6025-436d-ae55-06be5ff09eb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:37.738565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d3507-6025-436d-ae55-06be5ff09eb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:37.738828 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d3507-6025-436d-ae55-06be5ff09eb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:37.753626 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d3507-6025-436d-ae55-06be5ff09eb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:37.755157 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d3507-6025-436d-ae55-06be5ff09eb3 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:37.774117 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1381/2758] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:22:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:22:38.175242 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c8123-91cb-4c8e-9ebf-64007247801c None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:38.176881 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-f23c8123-91cb-4c8e-9ebf-64007247801c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:38.198827 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c8123-91cb-4c8e-9ebf-64007247801c None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:38.210599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238fe61-30e2-46bd-b589-f043ed8e87cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:38.211246 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238fe61-30e2-46bd-b589-f043ed8e87cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:38.211519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238fe61-30e2-46bd-b589-f043ed8e87cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:38.211789 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238fe61-30e2-46bd-b589-f043ed8e87cb admin admin] PATH_INFO: `/v3/projects/986d9add578a4688864fb9fcd016496d` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:38.213909 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0238fe61-30e2-46bd-b589-f043ed8e87cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=986d9add578a4688864fb9fcd016496d)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:38.215614 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0238fe61-30e2-46bd-b589-f043ed8e87cb admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:38.238163 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:38.297140 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-0238fe61-30e2-46bd-b589-f043ed8e87cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '986d9add578a4688864fb9fcd016496d', 'request_id': 'req-0238fe61-30e2-46bd-b589-f043ed8e87cb'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:38.307288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-0238fe61-30e2-46bd-b589-f043ed8e87cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '986d9add578a4688864fb9fcd016496d', 'request_id': 'req-0238fe61-30e2-46bd-b589-f043ed8e87cb'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:38.313551 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-0238fe61-30e2-46bd-b589-f043ed8e87cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '986d9add578a4688864fb9fcd016496d', 'request_id': 'req-0238fe61-30e2-46bd-b589-f043ed8e87cb'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:38.334405 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-0238fe61-30e2-46bd-b589-f043ed8e87cb admin admin] The token cache is being invalidate because project 986d9add578a4688864fb9fcd016496d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:22:38.334742 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-0238fe61-30e2-46bd-b589-f043ed8e87cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:38.335483 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-0238fe61-30e2-46bd-b589-f043ed8e87cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:38.337045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1378/2759] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:22:38 2022] DELETE /identity/v3/projects/986d9add578a4688864fb9fcd016496d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:22:38.341984 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d3507-6025-436d-ae55-06be5ff09eb3 service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:38.358784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d3507-6025-436d-ae55-06be5ff09eb3 service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:38.457245 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deddec2b-8519-4acb-ad41-904b1c8c272e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:38.458348 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddec2b-8519-4acb-ad41-904b1c8c272e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:38.458758 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddec2b-8519-4acb-ad41-904b1c8c272e admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:38.459069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddec2b-8519-4acb-ad41-904b1c8c272e admin admin] PATH_INFO: `/v3/users/33502be8431047db880c7842e51c7c9b` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:38.460768 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deddec2b-8519-4acb-ad41-904b1c8c272e admin admin] RBAC: Authorizing `identity:delete_user(user_id=33502be8431047db880c7842e51c7c9b)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:38.463240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deddec2b-8519-4acb-ad41-904b1c8c272e admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:38.539294 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-deddec2b-8519-4acb-ad41-904b1c8c272e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33502be8431047db880c7842e51c7c9b'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:38.562336 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-deddec2b-8519-4acb-ad41-904b1c8c272e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33502be8431047db880c7842e51c7c9b'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:38.572914 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-deddec2b-8519-4acb-ad41-904b1c8c272e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33502be8431047db880c7842e51c7c9b'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:38.582331 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:38.585870 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-deddec2b-8519-4acb-ad41-904b1c8c272e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33502be8431047db880c7842e51c7c9b'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:38.587099 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-deddec2b-8519-4acb-ad41-904b1c8c272e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33502be8431047db880c7842e51c7c9b'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:38.596894 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1382/2760] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:22:38 2022] DELETE /identity/v3/users/33502be8431047db880c7842e51c7c9b => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:22:39.018867 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238fe61-30e2-46bd-b589-f043ed8e87cb admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:39.020966 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-0238fe61-30e2-46bd-b589-f043ed8e87cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:39.039403 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238fe61-30e2-46bd-b589-f043ed8e87cb admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:39.125692 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9df41-1c11-4105-9dac-382845ae7391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:39.126730 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9df41-1c11-4105-9dac-382845ae7391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:39.127358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9df41-1c11-4105-9dac-382845ae7391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:39.127684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9df41-1c11-4105-9dac-382845ae7391 admin admin] PATH_INFO: `/v3/projects/decd7571e1484329b02f3d407d746f46` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:39.129471 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9df41-1c11-4105-9dac-382845ae7391 admin admin] RBAC: Authorizing `identity:delete_project(project_id=decd7571e1484329b02f3d407d746f46)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:39.132924 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9df41-1c11-4105-9dac-382845ae7391 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:39.202593 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-19c9df41-1c11-4105-9dac-382845ae7391 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'decd7571e1484329b02f3d407d746f46', 'request_id': 'req-19c9df41-1c11-4105-9dac-382845ae7391'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:39.210126 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-19c9df41-1c11-4105-9dac-382845ae7391 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'decd7571e1484329b02f3d407d746f46', 'request_id': 'req-19c9df41-1c11-4105-9dac-382845ae7391'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:39.219525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-19c9df41-1c11-4105-9dac-382845ae7391 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'decd7571e1484329b02f3d407d746f46', 'request_id': 'req-19c9df41-1c11-4105-9dac-382845ae7391'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:39.238585 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-19c9df41-1c11-4105-9dac-382845ae7391 admin admin] The token cache is being invalidate because project decd7571e1484329b02f3d407d746f46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:22:39.238869 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-19c9df41-1c11-4105-9dac-382845ae7391 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:39.239760 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-19c9df41-1c11-4105-9dac-382845ae7391 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:22:39.242450 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1379/2761] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:22:39 2022] DELETE /identity/v3/projects/decd7571e1484329b02f3d407d746f46 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:22:39.255954 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c6c51-a356-43fe-89d7-504ad90d4631 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:39.256205 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c6c51-a356-43fe-89d7-504ad90d4631 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:39.256419 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c6c51-a356-43fe-89d7-504ad90d4631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:39.279549 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-581c6c51-a356-43fe-89d7-504ad90d4631 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:22:39.281115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-581c6c51-a356-43fe-89d7-504ad90d4631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:39.363823 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1383/2762] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:22:39.372777 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72f533-a728-419a-89cf-5453781d0eae None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:39.373176 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72f533-a728-419a-89cf-5453781d0eae None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:39.373389 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72f533-a728-419a-89cf-5453781d0eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:39.399185 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-cf72f533-a728-419a-89cf-5453781d0eae None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:22:39.399698 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c6c51-a356-43fe-89d7-504ad90d4631 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:39.401095 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-581c6c51-a356-43fe-89d7-504ad90d4631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:39.401274 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-cf72f533-a728-419a-89cf-5453781d0eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:39.412582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1380/2763] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:22:39.417234 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72f533-a728-419a-89cf-5453781d0eae None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:39.427637 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72f533-a728-419a-89cf-5453781d0eae None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:39.432770 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fb7cf-74b8-4faa-83a2-fe26515fda3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:39.433431 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fb7cf-74b8-4faa-83a2-fe26515fda3b admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:39.433818 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fb7cf-74b8-4faa-83a2-fe26515fda3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:39.434007 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fb7cf-74b8-4faa-83a2-fe26515fda3b admin admin] PATH_INFO: `/v3/domains` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:39.434612 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6fb7cf-74b8-4faa-83a2-fe26515fda3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81781) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:22:39.434946 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6fb7cf-74b8-4faa-83a2-fe26515fda3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:39.436565 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6fb7cf-74b8-4faa-83a2-fe26515fda3b admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:39.452019 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1381/2764] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:22:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:22:39.458698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fb7cf-74b8-4faa-83a2-fe26515fda3b admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:39.478197 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fb7cf-74b8-4faa-83a2-fe26515fda3b admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:39.482469 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c6c51-a356-43fe-89d7-504ad90d4631 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:39.486118 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa7c57-cd7e-47b5-a8f0-346b2ab33e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:39.486533 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00387e41-7cdb-4d96-901e-7087f5dc6b92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:39.487072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa7c57-cd7e-47b5-a8f0-346b2ab33e5e admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:39.487192 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00387e41-7cdb-4d96-901e-7087f5dc6b92 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:39.487397 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00387e41-7cdb-4d96-901e-7087f5dc6b92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:39.487523 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa7c57-cd7e-47b5-a8f0-346b2ab33e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:39.487670 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00387e41-7cdb-4d96-901e-7087f5dc6b92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:39.487951 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa7c57-cd7e-47b5-a8f0-346b2ab33e5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:39.489000 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa7c57-cd7e-47b5-a8f0-346b2ab33e5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:39.489490 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-00387e41-7cdb-4d96-901e-7087f5dc6b92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:39.492208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa7c57-cd7e-47b5-a8f0-346b2ab33e5e admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:39.509788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:39.547294 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1382/2765] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:22:39 2022] POST /identity/v3/projects => generated 385 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:22:39.552414 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa7c57-cd7e-47b5-a8f0-346b2ab33e5e admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:39.616346 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00387e41-7cdb-4d96-901e-7087f5dc6b92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:39.618474 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00387e41-7cdb-4d96-901e-7087f5dc6b92 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:39.633110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1384/2766] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:22:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:22:39.645758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa7c57-cd7e-47b5-a8f0-346b2ab33e5e admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:39.654556 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274e03f-f38c-44b7-9ca8-771eaa9e4d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:39.655186 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274e03f-f38c-44b7-9ca8-771eaa9e4d79 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:39.655472 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274e03f-f38c-44b7-9ca8-771eaa9e4d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:39.655743 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274e03f-f38c-44b7-9ca8-771eaa9e4d79 admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:39.656393 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c274e03f-f38c-44b7-9ca8-771eaa9e4d79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:39.658412 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c274e03f-f38c-44b7-9ca8-771eaa9e4d79 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:39.682895 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1383/2767] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:22:39 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:22:39.693654 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00387e41-7cdb-4d96-901e-7087f5dc6b92 service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:39.706052 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00387e41-7cdb-4d96-901e-7087f5dc6b92 service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:39.711284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202b5d1-abbc-435f-b31e-603dc8f3e3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:39.711963 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202b5d1-abbc-435f-b31e-603dc8f3e3c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:39.712237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202b5d1-abbc-435f-b31e-603dc8f3e3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:39.712498 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202b5d1-abbc-435f-b31e-603dc8f3e3c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:39.713145 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3202b5d1-abbc-435f-b31e-603dc8f3e3c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:39.714789 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3202b5d1-abbc-435f-b31e-603dc8f3e3c0 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:39.731251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1385/2768] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:22:39.738810 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274e03f-f38c-44b7-9ca8-771eaa9e4d79 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:39.757962 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274e03f-f38c-44b7-9ca8-771eaa9e4d79 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:39.768486 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db144c-de02-4a19-b948-32b8c533cc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:39.769439 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db144c-de02-4a19-b948-32b8c533cc24 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:39.769970 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db144c-de02-4a19-b948-32b8c533cc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:39.770354 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db144c-de02-4a19-b948-32b8c533cc24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:39.771407 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db144c-de02-4a19-b948-32b8c533cc24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:39.774196 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db144c-de02-4a19-b948-32b8c533cc24 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:39.794064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1384/2769] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:22:39.799468 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202b5d1-abbc-435f-b31e-603dc8f3e3c0 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:39.814839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202b5d1-abbc-435f-b31e-603dc8f3e3c0 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:39.821257 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e544a-cc32-4ae0-ac76-a90c642e40e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:39.821957 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e544a-cc32-4ae0-ac76-a90c642e40e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:39.822244 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e544a-cc32-4ae0-ac76-a90c642e40e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:39.822510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e544a-cc32-4ae0-ac76-a90c642e40e7 admin admin] PATH_INFO: `/v3/projects/9f9f5a2386e2451f9a9b833859343595/users/fc11bab9d92c48e3938593035b837da1/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:39.846814 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e544a-cc32-4ae0-ac76-a90c642e40e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f9f5a2386e2451f9a9b833859343595, user_id=fc11bab9d92c48e3938593035b837da1, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:39.852423 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e544a-cc32-4ae0-ac76-a90c642e40e7 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:39.879035 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1386/2770] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:22:39 2022] PUT /identity/v3/projects/9f9f5a2386e2451f9a9b833859343595/users/fc11bab9d92c48e3938593035b837da1/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:22:39.884185 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1a55e-e25b-43f8-bf5f-d1ac6a5047f6 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:39.884572 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1a55e-e25b-43f8-bf5f-d1ac6a5047f6 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:39.884979 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1a55e-e25b-43f8-bf5f-d1ac6a5047f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:39.914588 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-90b1a55e-e25b-43f8-bf5f-d1ac6a5047f6 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:22:39.915878 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-90b1a55e-e25b-43f8-bf5f-d1ac6a5047f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:40.014028 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1385/2771] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:22:40.026729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e544a-cc32-4ae0-ac76-a90c642e40e7 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:40.028578 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-4a7e544a-cc32-4ae0-ac76-a90c642e40e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:40.035386 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:40.128459 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e544a-cc32-4ae0-ac76-a90c642e40e7 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:40.133496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1377a4-3819-4c35-9dba-ce1e72f45ce5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:40.134349 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1377a4-3819-4c35-9dba-ce1e72f45ce5 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:40.134771 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1377a4-3819-4c35-9dba-ce1e72f45ce5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:40.135154 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1377a4-3819-4c35-9dba-ce1e72f45ce5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:40.148913 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1377a4-3819-4c35-9dba-ce1e72f45ce5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:40.150951 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1377a4-3819-4c35-9dba-ce1e72f45ce5 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:40.163959 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1387/2772] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:22:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:22:45.193565 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940ef3b-90ac-47cf-9bc6-1599f95f6031 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:45.194104 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940ef3b-90ac-47cf-9bc6-1599f95f6031 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:45.194190 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940ef3b-90ac-47cf-9bc6-1599f95f6031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:45.229057 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-2940ef3b-90ac-47cf-9bc6-1599f95f6031 None None] MFA Rules not processed for user `fc11bab9d92c48e3938593035b837da1`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:22:45.230321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-2940ef3b-90ac-47cf-9bc6-1599f95f6031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:45.303353 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:45.312444 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1386/2773] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:22:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:22:45.314230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1377a4-3819-4c35-9dba-ce1e72f45ce5 service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:45.325392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1377a4-3819-4c35-9dba-ce1e72f45ce5 service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:45.331820 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b24c19-852f-45ba-ac29-187de56af98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:45.332575 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b24c19-852f-45ba-ac29-187de56af98f admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:45.332874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b24c19-852f-45ba-ac29-187de56af98f admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:45.333322 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b24c19-852f-45ba-ac29-187de56af98f admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:45.333927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b24c19-852f-45ba-ac29-187de56af98f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:45.335870 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b24c19-852f-45ba-ac29-187de56af98f admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:45.396763 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1388/2774] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:22:45 2022] POST /identity/v3/projects => generated 385 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:22:45.400837 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940ef3b-90ac-47cf-9bc6-1599f95f6031 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:45.419045 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940ef3b-90ac-47cf-9bc6-1599f95f6031 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:45.495678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e6000-ca95-46cc-83a3-597f978db17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:45.496397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e6000-ca95-46cc-83a3-597f978db17a admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:45.496742 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e6000-ca95-46cc-83a3-597f978db17a admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:45.497086 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e6000-ca95-46cc-83a3-597f978db17a admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:45.497851 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061e6000-ca95-46cc-83a3-597f978db17a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:45.500438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061e6000-ca95-46cc-83a3-597f978db17a admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:45.528739 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1387/2775] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:22:45 2022] POST /identity/v3/users => generated 299 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:22:45.534004 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b24c19-852f-45ba-ac29-187de56af98f admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:45.553037 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b24c19-852f-45ba-ac29-187de56af98f admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:45.562151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66942c-6142-4fae-9f5f-0212d6069a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:45.563921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66942c-6142-4fae-9f5f-0212d6069a97 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:45.564411 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66942c-6142-4fae-9f5f-0212d6069a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:45.564896 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66942c-6142-4fae-9f5f-0212d6069a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:45.566136 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66942c-6142-4fae-9f5f-0212d6069a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:45.569158 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66942c-6142-4fae-9f5f-0212d6069a97 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:45.593552 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1389/2776] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:22:45.598556 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e6000-ca95-46cc-83a3-597f978db17a admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:45.618335 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e6000-ca95-46cc-83a3-597f978db17a admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:45.627458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1873de4-de4b-46c2-929e-98feb235e5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:45.628425 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1873de4-de4b-46c2-929e-98feb235e5d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:45.628748 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1873de4-de4b-46c2-929e-98feb235e5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:45.629065 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1873de4-de4b-46c2-929e-98feb235e5d3 admin admin] PATH_INFO: `/v3/projects/79c093bab922455e9fc9bbf0b93fa278/users/1e501bd548764abe9c3ab9898f7a5d08/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:45.650383 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1873de4-de4b-46c2-929e-98feb235e5d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79c093bab922455e9fc9bbf0b93fa278, user_id=1e501bd548764abe9c3ab9898f7a5d08, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:45.653757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1873de4-de4b-46c2-929e-98feb235e5d3 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:45.677578 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1388/2777] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:22:45 2022] PUT /identity/v3/projects/79c093bab922455e9fc9bbf0b93fa278/users/1e501bd548764abe9c3ab9898f7a5d08/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:22:45.682359 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66942c-6142-4fae-9f5f-0212d6069a97 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:45.694817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66942c-6142-4fae-9f5f-0212d6069a97 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:45.781102 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237642c8-df9f-4396-b98a-b13993d8c55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:45.782740 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237642c8-df9f-4396-b98a-b13993d8c55e admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:45.783210 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237642c8-df9f-4396-b98a-b13993d8c55e admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:45.783493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237642c8-df9f-4396-b98a-b13993d8c55e admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:45.784317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237642c8-df9f-4396-b98a-b13993d8c55e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:45.786101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237642c8-df9f-4396-b98a-b13993d8c55e admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:45.804221 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1390/2778] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:22:45.805638 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1873de4-de4b-46c2-929e-98feb235e5d3 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:45.818023 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1873de4-de4b-46c2-929e-98feb235e5d3 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:45.831301 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc3a35-adbb-4f22-9af0-08be53ff6c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:45.832406 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc3a35-adbb-4f22-9af0-08be53ff6c19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:45.832534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc3a35-adbb-4f22-9af0-08be53ff6c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:45.832778 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc3a35-adbb-4f22-9af0-08be53ff6c19 admin admin] PATH_INFO: `/v3/projects/79c093bab922455e9fc9bbf0b93fa278/users/1e501bd548764abe9c3ab9898f7a5d08/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:45.835361 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc3a35-adbb-4f22-9af0-08be53ff6c19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79c093bab922455e9fc9bbf0b93fa278, user_id=1e501bd548764abe9c3ab9898f7a5d08, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:45.838723 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc3a35-adbb-4f22-9af0-08be53ff6c19 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:45.855876 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1389/2779] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:22:45 2022] PUT /identity/v3/projects/79c093bab922455e9fc9bbf0b93fa278/users/1e501bd548764abe9c3ab9898f7a5d08/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:22:51.008034 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdac84-d883-4afe-bb9d-5a7e46ca175f None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:51.008567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdac84-d883-4afe-bb9d-5a7e46ca175f None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:51.008677 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdac84-d883-4afe-bb9d-5a7e46ca175f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:51.040804 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-abcdac84-d883-4afe-bb9d-5a7e46ca175f None None] MFA Rules not processed for user `1e501bd548764abe9c3ab9898f7a5d08`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:22:51.042088 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-abcdac84-d883-4afe-bb9d-5a7e46ca175f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:51.112672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:51.128399 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1391/2780] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:22:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:22:51.136652 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc3a35-adbb-4f22-9af0-08be53ff6c19 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:51.138476 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-e5fc3a35-adbb-4f22-9af0-08be53ff6c19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:51.240331 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc3a35-adbb-4f22-9af0-08be53ff6c19 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:51.246873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b9e07-80a2-46eb-9335-8f73217be1b4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b64b4b172467692d02851b2955b81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:51.248511 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b9e07-80a2-46eb-9335-8f73217be1b4 service glance] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:51.248814 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b9e07-80a2-46eb-9335-8f73217be1b4 service glance] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:51.249138 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b9e07-80a2-46eb-9335-8f73217be1b4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:51.316617 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b9e07-80a2-46eb-9335-8f73217be1b4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:51.318099 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b9e07-80a2-46eb-9335-8f73217be1b4 service glance] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:51.328668 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1390/2781] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:22:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:22:51.503358 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d88372-e62d-4e8f-ba12-bcc89d79b0d0 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:51.503887 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d88372-e62d-4e8f-ba12-bcc89d79b0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:51.504474 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d88372-e62d-4e8f-ba12-bcc89d79b0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:51.532804 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-27d88372-e62d-4e8f-ba12-bcc89d79b0d0 None None] MFA Rules not processed for user `e0c76478b6694782b896cdf9cda32d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:22:51.534126 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-27d88372-e62d-4e8f-ba12-bcc89d79b0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:51.655613 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1392/2782] 10.222.0.73 () {58 vars in 1009 bytes} [Mon Aug 29 20:22:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:22:51.728419 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b9e07-80a2-46eb-9335-8f73217be1b4 service glance] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:51.730417 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9f7b9e07-80a2-46eb-9335-8f73217be1b4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:51.738077 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:51.833586 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b9e07-80a2-46eb-9335-8f73217be1b4 service glance] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:51.839540 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0103e4-41f1-4af5-a62a-ffd16c85f697 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92e30da8f394f3abc19b9e86ef126e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:51.840304 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0103e4-41f1-4af5-a62a-ffd16c85f697 service swift] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:51.840476 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0103e4-41f1-4af5-a62a-ffd16c85f697 service swift] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:51.840722 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0103e4-41f1-4af5-a62a-ffd16c85f697 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:51.851632 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0103e4-41f1-4af5-a62a-ffd16c85f697 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:51.853376 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0103e4-41f1-4af5-a62a-ffd16c85f697 service swift] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:51.865108 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1391/2783] 10.222.0.73 () {60 vars in 1384 bytes} [Mon Aug 29 20:22:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:23:01.176415 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeffcd7-1670-4862-96b5-491591d84595 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:01.177360 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeffcd7-1670-4862-96b5-491591d84595 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:01.177360 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeffcd7-1670-4862-96b5-491591d84595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:01.208458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-aaeffcd7-1670-4862-96b5-491591d84595 None None] MFA Rules not processed for user `fc11bab9d92c48e3938593035b837da1`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:23:01.209790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-aaeffcd7-1670-4862-96b5-491591d84595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:01.217243 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1393/2784] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:23:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:23:01.860748 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=30 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:01.956568 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0103e4-41f1-4af5-a62a-ffd16c85f697 service swift] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:02.052251 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0103e4-41f1-4af5-a62a-ffd16c85f697 service swift] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:02.055901 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de5312-3e1f-440a-89de-62512f53f983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:02.056516 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de5312-3e1f-440a-89de-62512f53f983 service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:02.056836 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de5312-3e1f-440a-89de-62512f53f983 service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:02.057075 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de5312-3e1f-440a-89de-62512f53f983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:02.059060 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-86de5312-3e1f-440a-89de-62512f53f983 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:02.160672 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de5312-3e1f-440a-89de-62512f53f983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:02.163525 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de5312-3e1f-440a-89de-62512f53f983 service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:02.184725 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1392/2785] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:23:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:23:03.922096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e370fe3-7bda-42ad-ab23-88c6133da5c4 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:03.922701 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e370fe3-7bda-42ad-ab23-88c6133da5c4 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:03.922701 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e370fe3-7bda-42ad-ab23-88c6133da5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:04.007188 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-9e370fe3-7bda-42ad-ab23-88c6133da5c4 None None] MFA Rules not processed for user `e0c76478b6694782b896cdf9cda32d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:23:04.008573 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-9e370fe3-7bda-42ad-ab23-88c6133da5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:04.034627 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1394/2786] 10.222.0.73 () {58 vars in 1009 bytes} [Mon Aug 29 20:23:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:23:04.053000 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de5312-3e1f-440a-89de-62512f53f983 service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:04.091134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de5312-3e1f-440a-89de-62512f53f983 service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:04.097902 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88798925-5ddd-4cbf-a9dc-2243a236e7ff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92e30da8f394f3abc19b9e86ef126e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:04.098592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88798925-5ddd-4cbf-a9dc-2243a236e7ff service swift] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:04.098774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88798925-5ddd-4cbf-a9dc-2243a236e7ff service swift] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:04.099113 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88798925-5ddd-4cbf-a9dc-2243a236e7ff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:04.112748 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88798925-5ddd-4cbf-a9dc-2243a236e7ff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:04.114971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88798925-5ddd-4cbf-a9dc-2243a236e7ff service swift] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:04.126559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1393/2787] 10.222.0.73 () {60 vars in 1384 bytes} [Mon Aug 29 20:23:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:23:14.031252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=49 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:14.120401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=64 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:19.174075 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e370fe3-7bda-42ad-ab23-88c6133da5c4 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:19.237384 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e370fe3-7bda-42ad-ab23-88c6133da5c4 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:19.241011 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59dac80-7f46-4ce6-be25-5c5cb6efcd78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:19.241621 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dac80-7f46-4ce6-be25-5c5cb6efcd78 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:19.241924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dac80-7f46-4ce6-be25-5c5cb6efcd78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:19.242308 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dac80-7f46-4ce6-be25-5c5cb6efcd78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:19.244765 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f59dac80-7f46-4ce6-be25-5c5cb6efcd78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:19.309777 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59dac80-7f46-4ce6-be25-5c5cb6efcd78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:19.311362 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59dac80-7f46-4ce6-be25-5c5cb6efcd78 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:19.321096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1395/2788] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:23:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:23:29.316839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=48 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:30.633403 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837afc0-d598-4df5-9311-4f8722dff47f None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:30.634026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837afc0-d598-4df5-9311-4f8722dff47f None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:30.634313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837afc0-d598-4df5-9311-4f8722dff47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:30.666509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-0837afc0-d598-4df5-9311-4f8722dff47f None None] MFA Rules not processed for user `8640600ca0d745cda547a1027c947089`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:23:30.668553 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-0837afc0-d598-4df5-9311-4f8722dff47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:30.681036 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1394/2789] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:23:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:23:31.186094 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59dac80-7f46-4ce6-be25-5c5cb6efcd78 service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:31.189493 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f59dac80-7f46-4ce6-be25-5c5cb6efcd78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:31.209069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59dac80-7f46-4ce6-be25-5c5cb6efcd78 service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:31.214270 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48c8d5-0ab9-4aa0-b8ed-fed32a4dd0c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:31.214865 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48c8d5-0ab9-4aa0-b8ed-fed32a4dd0c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:31.215483 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48c8d5-0ab9-4aa0-b8ed-fed32a4dd0c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:31.215965 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48c8d5-0ab9-4aa0-b8ed-fed32a4dd0c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:31.227287 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-6b48c8d5-0ab9-4aa0-b8ed-fed32a4dd0c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:31.249472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b48c8d5-0ab9-4aa0-b8ed-fed32a4dd0c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:31.253064 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b48c8d5-0ab9-4aa0-b8ed-fed32a4dd0c8 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:31.270286 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1396/2790] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:23:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:23:32.021381 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0837afc0-d598-4df5-9311-4f8722dff47f None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:32.023776 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-0837afc0-d598-4df5-9311-4f8722dff47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:32.126009 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0837afc0-d598-4df5-9311-4f8722dff47f None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:32.132473 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a96cfd-d023-4beb-9fdb-f365c9e549b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab2e318b2854c46bc7ed2151a44a2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:32.133115 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a96cfd-d023-4beb-9fdb-f365c9e549b1 service placement] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:32.133396 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a96cfd-d023-4beb-9fdb-f365c9e549b1 service placement] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:32.133669 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a96cfd-d023-4beb-9fdb-f365c9e549b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:32.135486 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-a0a96cfd-d023-4beb-9fdb-f365c9e549b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:32.148413 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a96cfd-d023-4beb-9fdb-f365c9e549b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:32.150036 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a96cfd-d023-4beb-9fdb-f365c9e549b1 service placement] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:32.160876 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1395/2791] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:23:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:23:35.757699 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cede51-0594-4be3-9bb3-4aa0433132bc None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:35.758329 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cede51-0594-4be3-9bb3-4aa0433132bc None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:35.758598 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cede51-0594-4be3-9bb3-4aa0433132bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:35.787675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-76cede51-0594-4be3-9bb3-4aa0433132bc None None] MFA Rules not processed for user `47d7f801e70d4e08b73a83451fd522b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:23:35.789482 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-76cede51-0594-4be3-9bb3-4aa0433132bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:35.871159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1397/2792] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:23:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:23:39.001566 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a96cfd-d023-4beb-9fdb-f365c9e549b1 service placement] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:39.008625 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-a0a96cfd-d023-4beb-9fdb-f365c9e549b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:39.034077 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a96cfd-d023-4beb-9fdb-f365c9e549b1 service placement] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:39.042664 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb5665-9abf-473b-b6ad-20f66f3ccd8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:39.044253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb5665-9abf-473b-b6ad-20f66f3ccd8c service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:39.044952 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb5665-9abf-473b-b6ad-20f66f3ccd8c service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:39.045747 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb5665-9abf-473b-b6ad-20f66f3ccd8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:39.049803 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-68fb5665-9abf-473b-b6ad-20f66f3ccd8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:39.089264 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb5665-9abf-473b-b6ad-20f66f3ccd8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:39.092986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb5665-9abf-473b-b6ad-20f66f3ccd8c service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:39.117336 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1396/2793] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:23:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:23:40.646450 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cede51-0594-4be3-9bb3-4aa0433132bc None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:40.667192 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cede51-0594-4be3-9bb3-4aa0433132bc None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:40.671833 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b853898-23f2-4fee-8003-40766e24373b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:40.673229 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b853898-23f2-4fee-8003-40766e24373b service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:40.678424 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b853898-23f2-4fee-8003-40766e24373b service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:40.679258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b853898-23f2-4fee-8003-40766e24373b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:40.684653 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-3b853898-23f2-4fee-8003-40766e24373b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:40.741991 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b853898-23f2-4fee-8003-40766e24373b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:40.745908 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b853898-23f2-4fee-8003-40766e24373b service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:40.775423 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1398/2794] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:23:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:23:41.886169 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb5665-9abf-473b-b6ad-20f66f3ccd8c service nova] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:41.948659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb5665-9abf-473b-b6ad-20f66f3ccd8c service nova] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:41.955396 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32b7d4-77d3-4bae-a1d8-5b9f2ed42a48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:41.956436 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32b7d4-77d3-4bae-a1d8-5b9f2ed42a48 service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:41.957157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32b7d4-77d3-4bae-a1d8-5b9f2ed42a48 service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:41.958880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32b7d4-77d3-4bae-a1d8-5b9f2ed42a48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:41.962469 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-4a32b7d4-77d3-4bae-a1d8-5b9f2ed42a48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:41.994443 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32b7d4-77d3-4bae-a1d8-5b9f2ed42a48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:41.997732 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32b7d4-77d3-4bae-a1d8-5b9f2ed42a48 service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:42.019595 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1397/2795] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:23:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:23:45.827837 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054df8b2-890e-455d-a5e3-0447afe245eb None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:45.828386 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054df8b2-890e-455d-a5e3-0447afe245eb None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:45.828645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054df8b2-890e-455d-a5e3-0447afe245eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:45.865801 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-054df8b2-890e-455d-a5e3-0447afe245eb None None] MFA Rules not processed for user `c7c8422c3d314adca25acf967ffd0998`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:23:45.867316 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-054df8b2-890e-455d-a5e3-0447afe245eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:45.957025 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1399/2796] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:23:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:23:46.880904 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32b7d4-77d3-4bae-a1d8-5b9f2ed42a48 service nova] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:46.897382 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32b7d4-77d3-4bae-a1d8-5b9f2ed42a48 service nova] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:46.901729 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6cde64-0e34-4cc6-8794-09d58076c79e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:46.902601 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cde64-0e34-4cc6-8794-09d58076c79e service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:46.903092 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cde64-0e34-4cc6-8794-09d58076c79e service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:46.903721 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cde64-0e34-4cc6-8794-09d58076c79e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:46.918669 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6cde64-0e34-4cc6-8794-09d58076c79e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:46.920382 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6cde64-0e34-4cc6-8794-09d58076c79e service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:46.934442 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1398/2797] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:23:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:23:48.726295 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054df8b2-890e-455d-a5e3-0447afe245eb None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:48.727960 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-054df8b2-890e-455d-a5e3-0447afe245eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:48.753127 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054df8b2-890e-455d-a5e3-0447afe245eb None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:48.760563 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00fbb9-6905-4196-a452-bd0038b8b009 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:48.761331 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00fbb9-6905-4196-a452-bd0038b8b009 service nova] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:48.761734 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00fbb9-6905-4196-a452-bd0038b8b009 service nova] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:48.762103 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00fbb9-6905-4196-a452-bd0038b8b009 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:48.764027 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-fb00fbb9-6905-4196-a452-bd0038b8b009 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:48.781656 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00fbb9-6905-4196-a452-bd0038b8b009 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:48.784427 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00fbb9-6905-4196-a452-bd0038b8b009 service nova] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:48.798339 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1400/2798] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:23:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:23:48.824308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6cde64-0e34-4cc6-8794-09d58076c79e service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:48.837453 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6cde64-0e34-4cc6-8794-09d58076c79e service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:48.855214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5e26c-8dcd-4c23-83f0-d930f106e3dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:48.856560 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5e26c-8dcd-4c23-83f0-d930f106e3dd service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:48.856914 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5e26c-8dcd-4c23-83f0-d930f106e3dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:48.857261 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5e26c-8dcd-4c23-83f0-d930f106e3dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:48.859237 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-6ff5e26c-8dcd-4c23-83f0-d930f106e3dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:48.871648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5e26c-8dcd-4c23-83f0-d930f106e3dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:48.873089 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5e26c-8dcd-4c23-83f0-d930f106e3dd service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:48.884228 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1399/2799] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:23:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:23:58.792156 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=68 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:58.878469 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=59 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:24:00.864600 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00fbb9-6905-4196-a452-bd0038b8b009 service nova] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:24:00.887126 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00fbb9-6905-4196-a452-bd0038b8b009 service nova] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:24:00.893816 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c388f-cd0c-4485-9df4-e69ac87387d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:24:00.895054 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c388f-cd0c-4485-9df4-e69ac87387d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:00.897304 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c388f-cd0c-4485-9df4-e69ac87387d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:00.898495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c388f-cd0c-4485-9df4-e69ac87387d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:00.902618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-aa4c388f-cd0c-4485-9df4-e69ac87387d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:24:01.008806 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c388f-cd0c-4485-9df4-e69ac87387d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:24:01.010730 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c388f-cd0c-4485-9df4-e69ac87387d3 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:24:01.025201 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1401/2800] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:24:00 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:24:11.019107 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=30 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:24:43.925300 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5e26c-8dcd-4c23-83f0-d930f106e3dd service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:24:43.926490 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-6ff5e26c-8dcd-4c23-83f0-d930f106e3dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:24:44.005086 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5e26c-8dcd-4c23-83f0-d930f106e3dd service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:24:44.012458 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18ce3a-3b66-4777-b45f-aa954b3a2751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:24:44.013370 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18ce3a-3b66-4777-b45f-aa954b3a2751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:44.013866 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18ce3a-3b66-4777-b45f-aa954b3a2751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:44.014281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18ce3a-3b66-4777-b45f-aa954b3a2751 admin admin] PATH_INFO: `/v3/users/47d7f801e70d4e08b73a83451fd522b1` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:44.015545 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b18ce3a-3b66-4777-b45f-aa954b3a2751 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47d7f801e70d4e08b73a83451fd522b1)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:24:44.017408 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b18ce3a-3b66-4777-b45f-aa954b3a2751 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:24:44.077734 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-1b18ce3a-3b66-4777-b45f-aa954b3a2751 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47d7f801e70d4e08b73a83451fd522b1'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:24:44.087159 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-1b18ce3a-3b66-4777-b45f-aa954b3a2751 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47d7f801e70d4e08b73a83451fd522b1'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:24:44.102736 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-1b18ce3a-3b66-4777-b45f-aa954b3a2751 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47d7f801e70d4e08b73a83451fd522b1'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:24:44.110391 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-1b18ce3a-3b66-4777-b45f-aa954b3a2751 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47d7f801e70d4e08b73a83451fd522b1'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:24:44.111141 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-1b18ce3a-3b66-4777-b45f-aa954b3a2751 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47d7f801e70d4e08b73a83451fd522b1'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:24:44.117007 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1400/2801] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:24:43 2022] DELETE /identity/v3/users/47d7f801e70d4e08b73a83451fd522b1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:24:44.298212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c388f-cd0c-4485-9df4-e69ac87387d3 service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:24:44.311989 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c388f-cd0c-4485-9df4-e69ac87387d3 service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:24:44.378388 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012d375-d9af-4085-9510-bebf7c0659e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:24:44.379051 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012d375-d9af-4085-9510-bebf7c0659e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:44.379335 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012d375-d9af-4085-9510-bebf7c0659e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:44.379611 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012d375-d9af-4085-9510-bebf7c0659e5 admin admin] PATH_INFO: `/v3/projects/3d6b1e6a93214e0fbe67b7dfb84f1043` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:44.380745 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012d375-d9af-4085-9510-bebf7c0659e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d6b1e6a93214e0fbe67b7dfb84f1043)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:24:44.382699 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012d375-d9af-4085-9510-bebf7c0659e5 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:24:44.433856 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-7012d375-d9af-4085-9510-bebf7c0659e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d6b1e6a93214e0fbe67b7dfb84f1043', 'request_id': 'req-7012d375-d9af-4085-9510-bebf7c0659e5'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:24:44.439513 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-7012d375-d9af-4085-9510-bebf7c0659e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d6b1e6a93214e0fbe67b7dfb84f1043', 'request_id': 'req-7012d375-d9af-4085-9510-bebf7c0659e5'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:24:44.446997 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-7012d375-d9af-4085-9510-bebf7c0659e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d6b1e6a93214e0fbe67b7dfb84f1043', 'request_id': 'req-7012d375-d9af-4085-9510-bebf7c0659e5'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:24:44.459209 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-7012d375-d9af-4085-9510-bebf7c0659e5 admin admin] The token cache is being invalidate because project 3d6b1e6a93214e0fbe67b7dfb84f1043 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:24:44.459496 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-7012d375-d9af-4085-9510-bebf7c0659e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:24:44.460133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-7012d375-d9af-4085-9510-bebf7c0659e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:24:44.461614 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1402/2802] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:24:44 2022] DELETE /identity/v3/projects/3d6b1e6a93214e0fbe67b7dfb84f1043 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:24:54.113475 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:24:54.456385 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:13.472485 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18ce3a-3b66-4777-b45f-aa954b3a2751 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:13.474608 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-1b18ce3a-3b66-4777-b45f-aa954b3a2751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:13.536366 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18ce3a-3b66-4777-b45f-aa954b3a2751 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:13.586267 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d79b7-7976-4afd-868a-1346660b87de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:13.586880 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d79b7-7976-4afd-868a-1346660b87de service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:13.587157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d79b7-7976-4afd-868a-1346660b87de service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:13.588030 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d79b7-7976-4afd-868a-1346660b87de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:13.590083 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d43d79b7-7976-4afd-868a-1346660b87de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:13.657873 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d79b7-7976-4afd-868a-1346660b87de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:13.659314 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d79b7-7976-4afd-868a-1346660b87de service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:13.671839 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1401/2803] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:25:17.092417 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012d375-d9af-4085-9510-bebf7c0659e5 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:17.094333 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-7012d375-d9af-4085-9510-bebf7c0659e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:17.109608 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012d375-d9af-4085-9510-bebf7c0659e5 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:17.114964 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba0225-42f6-4bd8-8886-8c31b32b89f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:17.115649 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba0225-42f6-4bd8-8886-8c31b32b89f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:17.115954 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba0225-42f6-4bd8-8886-8c31b32b89f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:17.116251 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba0225-42f6-4bd8-8886-8c31b32b89f5 admin admin] PATH_INFO: `/v3/users/c7c8422c3d314adca25acf967ffd0998` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:17.117532 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba0225-42f6-4bd8-8886-8c31b32b89f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7c8422c3d314adca25acf967ffd0998)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:17.119776 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba0225-42f6-4bd8-8886-8c31b32b89f5 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:17.186790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-23ba0225-42f6-4bd8-8886-8c31b32b89f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7c8422c3d314adca25acf967ffd0998'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:17.200295 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-23ba0225-42f6-4bd8-8886-8c31b32b89f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7c8422c3d314adca25acf967ffd0998'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:17.207817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-23ba0225-42f6-4bd8-8886-8c31b32b89f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7c8422c3d314adca25acf967ffd0998'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:17.215793 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-23ba0225-42f6-4bd8-8886-8c31b32b89f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7c8422c3d314adca25acf967ffd0998'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:17.216499 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-23ba0225-42f6-4bd8-8886-8c31b32b89f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7c8422c3d314adca25acf967ffd0998'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:17.224718 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1403/2804] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:25:17 2022] DELETE /identity/v3/users/c7c8422c3d314adca25acf967ffd0998 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:17.427669 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d79b7-7976-4afd-868a-1346660b87de service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:17.438773 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d79b7-7976-4afd-868a-1346660b87de service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:17.507314 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a428aa-f59d-4724-8ea9-1ba239b56050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:17.507988 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a428aa-f59d-4724-8ea9-1ba239b56050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:17.508286 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a428aa-f59d-4724-8ea9-1ba239b56050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:17.508559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a428aa-f59d-4724-8ea9-1ba239b56050 admin admin] PATH_INFO: `/v3/projects/8ee0b0ec6aaf40ba8f5ba9e3abdeebb8` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:17.510749 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a428aa-f59d-4724-8ea9-1ba239b56050 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ee0b0ec6aaf40ba8f5ba9e3abdeebb8)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:17.513725 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a428aa-f59d-4724-8ea9-1ba239b56050 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:17.522740 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:17.562323 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f3a428aa-f59d-4724-8ea9-1ba239b56050 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ee0b0ec6aaf40ba8f5ba9e3abdeebb8', 'request_id': 'req-f3a428aa-f59d-4724-8ea9-1ba239b56050'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:17.568738 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f3a428aa-f59d-4724-8ea9-1ba239b56050 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ee0b0ec6aaf40ba8f5ba9e3abdeebb8', 'request_id': 'req-f3a428aa-f59d-4724-8ea9-1ba239b56050'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:17.575019 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f3a428aa-f59d-4724-8ea9-1ba239b56050 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ee0b0ec6aaf40ba8f5ba9e3abdeebb8', 'request_id': 'req-f3a428aa-f59d-4724-8ea9-1ba239b56050'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:17.588944 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f3a428aa-f59d-4724-8ea9-1ba239b56050 admin admin] The token cache is being invalidate because project 8ee0b0ec6aaf40ba8f5ba9e3abdeebb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:25:17.589233 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f3a428aa-f59d-4724-8ea9-1ba239b56050 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:17.589913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-f3a428aa-f59d-4724-8ea9-1ba239b56050 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:17.591427 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1402/2805] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:25:17 2022] DELETE /identity/v3/projects/8ee0b0ec6aaf40ba8f5ba9e3abdeebb8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:17.597064 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef3182-9ef8-4709-bcdf-d7615bc02aa3 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:17.597565 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef3182-9ef8-4709-bcdf-d7615bc02aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:17.597874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef3182-9ef8-4709-bcdf-d7615bc02aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:17.618904 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-5bef3182-9ef8-4709-bcdf-d7615bc02aa3 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:17.620650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-5bef3182-9ef8-4709-bcdf-d7615bc02aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:17.688211 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1404/2806] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:17.696999 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c3bdc-fb53-4539-8edd-13e1a0b7718f None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:17.697307 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c3bdc-fb53-4539-8edd-13e1a0b7718f None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:17.697583 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c3bdc-fb53-4539-8edd-13e1a0b7718f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:17.719103 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-925c3bdc-fb53-4539-8edd-13e1a0b7718f None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:17.720514 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-925c3bdc-fb53-4539-8edd-13e1a0b7718f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:17.729238 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1403/2807] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:17.734096 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef3182-9ef8-4709-bcdf-d7615bc02aa3 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:17.745315 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef3182-9ef8-4709-bcdf-d7615bc02aa3 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:17.750028 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3f261-cccc-4994-a032-cd8d376a5db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:17.750656 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3f261-cccc-4994-a032-cd8d376a5db6 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:17.750938 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3f261-cccc-4994-a032-cd8d376a5db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:17.751207 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3f261-cccc-4994-a032-cd8d376a5db6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:17.751845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3f261-cccc-4994-a032-cd8d376a5db6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:25:17.752171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3f261-cccc-4994-a032-cd8d376a5db6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:17.753790 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3f261-cccc-4994-a032-cd8d376a5db6 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:17.766695 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1405/2808] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:25:17.771484 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c3bdc-fb53-4539-8edd-13e1a0b7718f None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:17.782519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c3bdc-fb53-4539-8edd-13e1a0b7718f None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:17.788139 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7b7e6-c751-4714-8f20-f0aa7b069538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:17.788756 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7b7e6-c751-4714-8f20-f0aa7b069538 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:17.789038 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7b7e6-c751-4714-8f20-f0aa7b069538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:17.789311 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7b7e6-c751-4714-8f20-f0aa7b069538 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:17.790182 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb7b7e6-c751-4714-8f20-f0aa7b069538 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:17.792102 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb7b7e6-c751-4714-8f20-f0aa7b069538 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:17.820943 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1404/2809] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:25:17 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:25:17.825881 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3f261-cccc-4994-a032-cd8d376a5db6 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:17.840286 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3f261-cccc-4994-a032-cd8d376a5db6 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:17.898401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329a208-36cd-401b-ac8b-42ba46e2e543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:17.899154 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329a208-36cd-401b-ac8b-42ba46e2e543 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:17.900129 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329a208-36cd-401b-ac8b-42ba46e2e543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:17.900472 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329a208-36cd-401b-ac8b-42ba46e2e543 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:17.901420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329a208-36cd-401b-ac8b-42ba46e2e543 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:17.903508 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329a208-36cd-401b-ac8b-42ba46e2e543 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:17.912114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:17.927736 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1406/2810] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:25:17 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:25:17.932728 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7b7e6-c751-4714-8f20-f0aa7b069538 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:17.943576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7b7e6-c751-4714-8f20-f0aa7b069538 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:17.949303 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb564d2-3011-4acc-b99c-6043b5c1c155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:17.949947 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb564d2-3011-4acc-b99c-6043b5c1c155 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:17.950225 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb564d2-3011-4acc-b99c-6043b5c1c155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:17.950494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb564d2-3011-4acc-b99c-6043b5c1c155 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:17.951130 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb564d2-3011-4acc-b99c-6043b5c1c155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:17.952734 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb564d2-3011-4acc-b99c-6043b5c1c155 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:17.964276 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1405/2811] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:25:17.968306 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329a208-36cd-401b-ac8b-42ba46e2e543 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:17.978743 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329a208-36cd-401b-ac8b-42ba46e2e543 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:17.983400 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731db9e-6744-472f-835f-4510a92bf09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:17.984001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731db9e-6744-472f-835f-4510a92bf09e admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:17.984275 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731db9e-6744-472f-835f-4510a92bf09e admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:17.984544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731db9e-6744-472f-835f-4510a92bf09e admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:17.985183 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c731db9e-6744-472f-835f-4510a92bf09e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:17.986821 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c731db9e-6744-472f-835f-4510a92bf09e admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:17.999207 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1407/2812] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:25:18.003664 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb564d2-3011-4acc-b99c-6043b5c1c155 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:18.014421 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb564d2-3011-4acc-b99c-6043b5c1c155 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:18.019047 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfc554-14d6-47ac-85c5-58cd7fe800c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:18.019689 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfc554-14d6-47ac-85c5-58cd7fe800c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:18.019962 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfc554-14d6-47ac-85c5-58cd7fe800c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:18.020229 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfc554-14d6-47ac-85c5-58cd7fe800c8 admin admin] PATH_INFO: `/v3/projects/b635e4623a1247498404d5d63917f30a/users/4a2b83905f844d97a229d443a05e5bef/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:18.036423 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cfc554-14d6-47ac-85c5-58cd7fe800c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b635e4623a1247498404d5d63917f30a, user_id=4a2b83905f844d97a229d443a05e5bef, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:18.039126 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cfc554-14d6-47ac-85c5-58cd7fe800c8 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:18.053259 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1406/2813] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:25:18 2022] PUT /identity/v3/projects/b635e4623a1247498404d5d63917f30a/users/4a2b83905f844d97a229d443a05e5bef/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:18.065082 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964afa3-f675-4167-99f7-7d479e79bfc8 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:18.065359 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964afa3-f675-4167-99f7-7d479e79bfc8 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:18.065630 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964afa3-f675-4167-99f7-7d479e79bfc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:18.102151 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-4964afa3-f675-4167-99f7-7d479e79bfc8 None None] MFA Rules not processed for user `4a2b83905f844d97a229d443a05e5bef`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:18.103540 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-4964afa3-f675-4167-99f7-7d479e79bfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:18.171058 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1408/2814] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:25:18.182985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfc554-14d6-47ac-85c5-58cd7fe800c8 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:18.185396 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-04cfc554-14d6-47ac-85c5-58cd7fe800c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:18.261650 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfc554-14d6-47ac-85c5-58cd7fe800c8 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:18.267274 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c6fe7-f7d4-422d-a5ce-4af1136eef58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95cf77a92a174ff8b0503baf5c7e63be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:18.267882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c6fe7-f7d4-422d-a5ce-4af1136eef58 service nova] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:18.268164 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c6fe7-f7d4-422d-a5ce-4af1136eef58 service nova] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:18.268439 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c6fe7-f7d4-422d-a5ce-4af1136eef58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:18.276916 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c6fe7-f7d4-422d-a5ce-4af1136eef58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:18.278854 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c6fe7-f7d4-422d-a5ce-4af1136eef58 service nova] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:18.287776 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1407/2815] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:21.016481 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964afa3-f675-4167-99f7-7d479e79bfc8 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:21.078784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964afa3-f675-4167-99f7-7d479e79bfc8 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:21.083366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57bae97-88db-4ef0-bec0-e26bfdfa52d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:21.083994 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bae97-88db-4ef0-bec0-e26bfdfa52d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:21.084276 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bae97-88db-4ef0-bec0-e26bfdfa52d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:21.084548 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bae97-88db-4ef0-bec0-e26bfdfa52d3 admin admin] PATH_INFO: `/v3/users/4a2b83905f844d97a229d443a05e5bef` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:21.085693 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57bae97-88db-4ef0-bec0-e26bfdfa52d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a2b83905f844d97a229d443a05e5bef)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:21.088598 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57bae97-88db-4ef0-bec0-e26bfdfa52d3 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:21.137977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-f57bae97-88db-4ef0-bec0-e26bfdfa52d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a2b83905f844d97a229d443a05e5bef'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:21.149840 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-f57bae97-88db-4ef0-bec0-e26bfdfa52d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a2b83905f844d97a229d443a05e5bef'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:21.153788 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:21.158249 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-f57bae97-88db-4ef0-bec0-e26bfdfa52d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a2b83905f844d97a229d443a05e5bef'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:21.167627 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-f57bae97-88db-4ef0-bec0-e26bfdfa52d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a2b83905f844d97a229d443a05e5bef'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:21.168273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-f57bae97-88db-4ef0-bec0-e26bfdfa52d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a2b83905f844d97a229d443a05e5bef'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:21.174133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1409/2816] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:25:21 2022] DELETE /identity/v3/users/4a2b83905f844d97a229d443a05e5bef => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:21.178640 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64212ab9-af2d-4ee0-af27-e2599dd0d9c0 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:21.179110 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64212ab9-af2d-4ee0-af27-e2599dd0d9c0 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:21.179225 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64212ab9-af2d-4ee0-af27-e2599dd0d9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:21.201959 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-64212ab9-af2d-4ee0-af27-e2599dd0d9c0 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:21.203198 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-64212ab9-af2d-4ee0-af27-e2599dd0d9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:21.215368 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:21.266026 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1408/2817] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:21.276223 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57bae97-88db-4ef0-bec0-e26bfdfa52d3 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:21.277667 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f57bae97-88db-4ef0-bec0-e26bfdfa52d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:21.336364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57bae97-88db-4ef0-bec0-e26bfdfa52d3 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:21.340248 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0bd59d-277d-4ca5-a2b4-f4235381c115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:21.340845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bd59d-277d-4ca5-a2b4-f4235381c115 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:21.341123 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bd59d-277d-4ca5-a2b4-f4235381c115 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:21.341394 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bd59d-277d-4ca5-a2b4-f4235381c115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:21.350533 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0bd59d-277d-4ca5-a2b4-f4235381c115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:21.351943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0bd59d-277d-4ca5-a2b4-f4235381c115 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:21.361379 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1410/2818] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:21.639382 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64212ab9-af2d-4ee0-af27-e2599dd0d9c0 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:21.641484 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-64212ab9-af2d-4ee0-af27-e2599dd0d9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:21.656048 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64212ab9-af2d-4ee0-af27-e2599dd0d9c0 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:21.660662 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f92099-f729-4721-9a4e-f11c60b5fd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:21.661293 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f92099-f729-4721-9a4e-f11c60b5fd94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:21.661571 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f92099-f729-4721-9a4e-f11c60b5fd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:21.661898 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f92099-f729-4721-9a4e-f11c60b5fd94 admin admin] PATH_INFO: `/v3/projects/b635e4623a1247498404d5d63917f30a` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:21.663155 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f92099-f729-4721-9a4e-f11c60b5fd94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b635e4623a1247498404d5d63917f30a)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:21.665486 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f92099-f729-4721-9a4e-f11c60b5fd94 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:21.713240 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-34f92099-f729-4721-9a4e-f11c60b5fd94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b635e4623a1247498404d5d63917f30a', 'request_id': 'req-34f92099-f729-4721-9a4e-f11c60b5fd94'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:21.719494 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-34f92099-f729-4721-9a4e-f11c60b5fd94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b635e4623a1247498404d5d63917f30a', 'request_id': 'req-34f92099-f729-4721-9a4e-f11c60b5fd94'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:21.725129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-34f92099-f729-4721-9a4e-f11c60b5fd94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b635e4623a1247498404d5d63917f30a', 'request_id': 'req-34f92099-f729-4721-9a4e-f11c60b5fd94'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:21.738468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-34f92099-f729-4721-9a4e-f11c60b5fd94 admin admin] The token cache is being invalidate because project b635e4623a1247498404d5d63917f30a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:25:21.738757 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-34f92099-f729-4721-9a4e-f11c60b5fd94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:21.739372 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-34f92099-f729-4721-9a4e-f11c60b5fd94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:21.740845 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1409/2819] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:25:21 2022] DELETE /identity/v3/projects/b635e4623a1247498404d5d63917f30a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:21.746760 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d146cd9-04d8-4680-aa8b-0c2730e2c014 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:21.747045 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d146cd9-04d8-4680-aa8b-0c2730e2c014 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:21.747317 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d146cd9-04d8-4680-aa8b-0c2730e2c014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:21.766876 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-9d146cd9-04d8-4680-aa8b-0c2730e2c014 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:21.767997 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-9d146cd9-04d8-4680-aa8b-0c2730e2c014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:21.836714 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1411/2820] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:21.845488 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc7334-3983-49b1-a3f5-72c3da109483 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:21.845799 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc7334-3983-49b1-a3f5-72c3da109483 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:21.846319 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc7334-3983-49b1-a3f5-72c3da109483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:21.866407 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-d8dc7334-3983-49b1-a3f5-72c3da109483 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:21.867596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-d8dc7334-3983-49b1-a3f5-72c3da109483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:21.875862 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1410/2821] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:21.880484 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d146cd9-04d8-4680-aa8b-0c2730e2c014 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:21.890497 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d146cd9-04d8-4680-aa8b-0c2730e2c014 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:21.895142 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6a1c2-22e0-4295-89fa-a5b5d5fa7069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:21.895734 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6a1c2-22e0-4295-89fa-a5b5d5fa7069 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:21.896009 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6a1c2-22e0-4295-89fa-a5b5d5fa7069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:21.896278 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6a1c2-22e0-4295-89fa-a5b5d5fa7069 admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:21.896909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a6a1c2-22e0-4295-89fa-a5b5d5fa7069 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:25:21.897233 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a6a1c2-22e0-4295-89fa-a5b5d5fa7069 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:21.898854 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a6a1c2-22e0-4295-89fa-a5b5d5fa7069 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:21.912169 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1412/2822] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:25:21.917735 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc7334-3983-49b1-a3f5-72c3da109483 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:21.927833 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc7334-3983-49b1-a3f5-72c3da109483 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:21.933334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7946c-cd44-459c-b99c-32c9dbf6c287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:21.933971 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7946c-cd44-459c-b99c-32c9dbf6c287 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:21.934288 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7946c-cd44-459c-b99c-32c9dbf6c287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:21.934588 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7946c-cd44-459c-b99c-32c9dbf6c287 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:21.935476 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7946c-cd44-459c-b99c-32c9dbf6c287 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:21.937361 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7946c-cd44-459c-b99c-32c9dbf6c287 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:21.965352 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1411/2823] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:25:21 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:25:21.969821 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6a1c2-22e0-4295-89fa-a5b5d5fa7069 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:21.981030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6a1c2-22e0-4295-89fa-a5b5d5fa7069 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:22.048751 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df2677-46c7-4229-bc1e-251a03373fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:22.049466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df2677-46c7-4229-bc1e-251a03373fdf admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:22.049619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df2677-46c7-4229-bc1e-251a03373fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:22.049922 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df2677-46c7-4229-bc1e-251a03373fdf admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:22.050569 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df2677-46c7-4229-bc1e-251a03373fdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:22.052487 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df2677-46c7-4229-bc1e-251a03373fdf admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:22.075531 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1413/2824] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:25:21 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:25:22.079696 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7946c-cd44-459c-b99c-32c9dbf6c287 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:22.092349 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7946c-cd44-459c-b99c-32c9dbf6c287 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:22.097383 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c36f16-298a-407e-b61e-6190e07dc664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:22.098031 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c36f16-298a-407e-b61e-6190e07dc664 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:22.098317 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c36f16-298a-407e-b61e-6190e07dc664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:22.098589 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c36f16-298a-407e-b61e-6190e07dc664 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:22.099234 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c36f16-298a-407e-b61e-6190e07dc664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:22.100826 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c36f16-298a-407e-b61e-6190e07dc664 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:22.112667 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1412/2825] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:25:22.116921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df2677-46c7-4229-bc1e-251a03373fdf admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:22.127615 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df2677-46c7-4229-bc1e-251a03373fdf admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:22.132233 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa11f8-c83f-44f8-84b6-abb76119ff6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:22.132830 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa11f8-c83f-44f8-84b6-abb76119ff6e admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:22.133106 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa11f8-c83f-44f8-84b6-abb76119ff6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:22.133376 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa11f8-c83f-44f8-84b6-abb76119ff6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:22.134353 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa11f8-c83f-44f8-84b6-abb76119ff6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:22.136179 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa11f8-c83f-44f8-84b6-abb76119ff6e admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:22.137548 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:22.151064 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1414/2826] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:25:22.154029 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c36f16-298a-407e-b61e-6190e07dc664 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:22.164487 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c36f16-298a-407e-b61e-6190e07dc664 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:22.170042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e47f3-74e1-4ad5-afc3-c09f3d3926ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:22.170698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e47f3-74e1-4ad5-afc3-c09f3d3926ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:22.170979 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e47f3-74e1-4ad5-afc3-c09f3d3926ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:22.171253 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e47f3-74e1-4ad5-afc3-c09f3d3926ad admin admin] PATH_INFO: `/v3/projects/8ab78d41be794a73a4bbaa7c0c934b85/users/0f2ff201e0b243b3850aa53a5f20f308/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:22.187452 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e47f3-74e1-4ad5-afc3-c09f3d3926ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ab78d41be794a73a4bbaa7c0c934b85, user_id=0f2ff201e0b243b3850aa53a5f20f308, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:22.190447 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e47f3-74e1-4ad5-afc3-c09f3d3926ad admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:22.206974 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1413/2827] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:25:22 2022] PUT /identity/v3/projects/8ab78d41be794a73a4bbaa7c0c934b85/users/0f2ff201e0b243b3850aa53a5f20f308/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:22.218227 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a3e8e-4171-4523-bbdc-fbc7bf39c6e5 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:22.218509 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a3e8e-4171-4523-bbdc-fbc7bf39c6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:22.218783 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a3e8e-4171-4523-bbdc-fbc7bf39c6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:22.249134 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-b49a3e8e-4171-4523-bbdc-fbc7bf39c6e5 None None] MFA Rules not processed for user `0f2ff201e0b243b3850aa53a5f20f308`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:22.250383 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-b49a3e8e-4171-4523-bbdc-fbc7bf39c6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:22.315974 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1415/2828] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:22.320398 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e47f3-74e1-4ad5-afc3-c09f3d3926ad admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:22.363905 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:22.387496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e47f3-74e1-4ad5-afc3-c09f3d3926ad admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:22.392427 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d572da-da39-4b73-bb9d-10c1ca78559a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:22.393032 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d572da-da39-4b73-bb9d-10c1ca78559a admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:22.393316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d572da-da39-4b73-bb9d-10c1ca78559a admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:22.393659 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d572da-da39-4b73-bb9d-10c1ca78559a admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:22.394241 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d572da-da39-4b73-bb9d-10c1ca78559a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:22.396125 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d572da-da39-4b73-bb9d-10c1ca78559a admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:22.425181 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1414/2829] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:25:22 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:25:22.429288 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a3e8e-4171-4523-bbdc-fbc7bf39c6e5 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:22.441449 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a3e8e-4171-4523-bbdc-fbc7bf39c6e5 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:22.499273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607823f-8002-436e-ac5b-9977ef54104b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:22.499878 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607823f-8002-436e-ac5b-9977ef54104b admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:22.500152 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607823f-8002-436e-ac5b-9977ef54104b admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:22.500421 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607823f-8002-436e-ac5b-9977ef54104b admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:22.501049 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607823f-8002-436e-ac5b-9977ef54104b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:22.502987 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607823f-8002-436e-ac5b-9977ef54104b admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:22.526169 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1416/2830] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:25:22 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:25:22.530462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d572da-da39-4b73-bb9d-10c1ca78559a admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:22.543507 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d572da-da39-4b73-bb9d-10c1ca78559a admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:22.548515 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da247f64-985f-44b2-abbb-ae3b7a7eb536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:22.549183 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da247f64-985f-44b2-abbb-ae3b7a7eb536 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:22.549621 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da247f64-985f-44b2-abbb-ae3b7a7eb536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:22.549996 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da247f64-985f-44b2-abbb-ae3b7a7eb536 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:22.550691 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da247f64-985f-44b2-abbb-ae3b7a7eb536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:22.552422 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da247f64-985f-44b2-abbb-ae3b7a7eb536 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:22.565809 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1415/2831] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:25:22.570168 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607823f-8002-436e-ac5b-9977ef54104b admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:22.580692 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607823f-8002-436e-ac5b-9977ef54104b admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:22.585340 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8aba0-2b10-4c11-b04f-bf35c6d7a30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:22.586086 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8aba0-2b10-4c11-b04f-bf35c6d7a30e admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:22.586417 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8aba0-2b10-4c11-b04f-bf35c6d7a30e admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:22.587417 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8aba0-2b10-4c11-b04f-bf35c6d7a30e admin admin] PATH_INFO: `/v3/projects/3002ca775c0d4d8ea85c5e35751fcc19/users/746130caf16647708c87b65a165e3207/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:22.606762 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8aba0-2b10-4c11-b04f-bf35c6d7a30e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3002ca775c0d4d8ea85c5e35751fcc19, user_id=746130caf16647708c87b65a165e3207, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:22.609475 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8aba0-2b10-4c11-b04f-bf35c6d7a30e admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:22.625734 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1417/2832] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:25:22 2022] PUT /identity/v3/projects/3002ca775c0d4d8ea85c5e35751fcc19/users/746130caf16647708c87b65a165e3207/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:22.629614 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da247f64-985f-44b2-abbb-ae3b7a7eb536 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:22.644113 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da247f64-985f-44b2-abbb-ae3b7a7eb536 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:22.713261 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce499c-4607-474d-862f-29ddff138b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:22.713888 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce499c-4607-474d-862f-29ddff138b05 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:22.714172 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce499c-4607-474d-862f-29ddff138b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:22.714447 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce499c-4607-474d-862f-29ddff138b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:22.715095 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce499c-4607-474d-862f-29ddff138b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:22.716685 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce499c-4607-474d-862f-29ddff138b05 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:22.727420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1416/2833] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:25:22.732152 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8aba0-2b10-4c11-b04f-bf35c6d7a30e admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:22.743054 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8aba0-2b10-4c11-b04f-bf35c6d7a30e admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:22.747923 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f758a-5c9b-4e0e-87d8-5a5d4f1d6d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:22.748591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f758a-5c9b-4e0e-87d8-5a5d4f1d6d94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:22.748896 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f758a-5c9b-4e0e-87d8-5a5d4f1d6d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:22.749172 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f758a-5c9b-4e0e-87d8-5a5d4f1d6d94 admin admin] PATH_INFO: `/v3/projects/3002ca775c0d4d8ea85c5e35751fcc19/users/746130caf16647708c87b65a165e3207/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:22.751143 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f758a-5c9b-4e0e-87d8-5a5d4f1d6d94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3002ca775c0d4d8ea85c5e35751fcc19, user_id=746130caf16647708c87b65a165e3207, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:22.753853 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f758a-5c9b-4e0e-87d8-5a5d4f1d6d94 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:22.764893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1418/2834] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:25:22 2022] PUT /identity/v3/projects/3002ca775c0d4d8ea85c5e35751fcc19/users/746130caf16647708c87b65a165e3207/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:22.777555 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0f89a-bf45-4902-a4f8-c78448258c8c None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:22.777868 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0f89a-bf45-4902-a4f8-c78448258c8c None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:22.778143 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0f89a-bf45-4902-a4f8-c78448258c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:22.809020 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-71b0f89a-bf45-4902-a4f8-c78448258c8c None None] MFA Rules not processed for user `746130caf16647708c87b65a165e3207`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:22.810274 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-71b0f89a-bf45-4902-a4f8-c78448258c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:22.889613 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1417/2835] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:22 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:22.902212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f758a-5c9b-4e0e-87d8-5a5d4f1d6d94 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:22.904060 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-3b7f758a-5c9b-4e0e-87d8-5a5d4f1d6d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:22.954523 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:22.971569 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f758a-5c9b-4e0e-87d8-5a5d4f1d6d94 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:22.975187 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b1529-97b7-40fd-bd95-3bf03464943d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:22.975794 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b1529-97b7-40fd-bd95-3bf03464943d service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:22.976066 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b1529-97b7-40fd-bd95-3bf03464943d service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:22.976332 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b1529-97b7-40fd-bd95-3bf03464943d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:22.986618 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b1529-97b7-40fd-bd95-3bf03464943d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:22.988074 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b1529-97b7-40fd-bd95-3bf03464943d service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:22.999567 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1419/2836] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:23.103078 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0f89a-bf45-4902-a4f8-c78448258c8c None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:23.115231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0f89a-bf45-4902-a4f8-c78448258c8c None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:23.121348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7905285-a1ed-4d22-9773-fa0b2b669960 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:23.122393 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7905285-a1ed-4d22-9773-fa0b2b669960 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:23.122874 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7905285-a1ed-4d22-9773-fa0b2b669960 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:23.123424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7905285-a1ed-4d22-9773-fa0b2b669960 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:23.174798 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:23.213752 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7905285-a1ed-4d22-9773-fa0b2b669960 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:23.215940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7905285-a1ed-4d22-9773-fa0b2b669960 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:23.230624 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1418/2837] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:32.994907 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=9 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:33.225215 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=13 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:33.514685 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b1529-97b7-40fd-bd95-3bf03464943d service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:33.596623 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b1529-97b7-40fd-bd95-3bf03464943d service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:33.603307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364255fb-21ce-4bb7-9c1e-4d6ae2124f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:33.604710 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364255fb-21ce-4bb7-9c1e-4d6ae2124f34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:33.605021 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364255fb-21ce-4bb7-9c1e-4d6ae2124f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:33.605302 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364255fb-21ce-4bb7-9c1e-4d6ae2124f34 admin admin] PATH_INFO: `/v3/users/0f2ff201e0b243b3850aa53a5f20f308` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:33.606700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364255fb-21ce-4bb7-9c1e-4d6ae2124f34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f2ff201e0b243b3850aa53a5f20f308)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:33.620965 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364255fb-21ce-4bb7-9c1e-4d6ae2124f34 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:33.682318 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-364255fb-21ce-4bb7-9c1e-4d6ae2124f34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f2ff201e0b243b3850aa53a5f20f308'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:33.700042 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-364255fb-21ce-4bb7-9c1e-4d6ae2124f34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f2ff201e0b243b3850aa53a5f20f308'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:33.712189 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-364255fb-21ce-4bb7-9c1e-4d6ae2124f34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f2ff201e0b243b3850aa53a5f20f308'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:33.729995 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-364255fb-21ce-4bb7-9c1e-4d6ae2124f34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f2ff201e0b243b3850aa53a5f20f308'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:33.730879 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-364255fb-21ce-4bb7-9c1e-4d6ae2124f34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f2ff201e0b243b3850aa53a5f20f308'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:33.738257 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1420/2838] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:25:33 2022] DELETE /identity/v3/users/0f2ff201e0b243b3850aa53a5f20f308 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:33.743615 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b627486-c0aa-4e5c-95f6-13acba6ba7ee None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:33.743900 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b627486-c0aa-4e5c-95f6-13acba6ba7ee None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:33.744167 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b627486-c0aa-4e5c-95f6-13acba6ba7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:33.769137 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-6b627486-c0aa-4e5c-95f6-13acba6ba7ee None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:33.770438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-6b627486-c0aa-4e5c-95f6-13acba6ba7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:33.854685 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1419/2839] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:33.866364 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364255fb-21ce-4bb7-9c1e-4d6ae2124f34 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:33.868576 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-364255fb-21ce-4bb7-9c1e-4d6ae2124f34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:33.937181 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364255fb-21ce-4bb7-9c1e-4d6ae2124f34 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:33.941003 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871edcf-4adc-419f-aa17-e480663cb9be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:33.942368 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871edcf-4adc-419f-aa17-e480663cb9be service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:33.942368 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871edcf-4adc-419f-aa17-e480663cb9be service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:33.942368 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871edcf-4adc-419f-aa17-e480663cb9be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:33.954927 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5871edcf-4adc-419f-aa17-e480663cb9be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:33.956640 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5871edcf-4adc-419f-aa17-e480663cb9be service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:33.968423 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1421/2840] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:34.313818 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b627486-c0aa-4e5c-95f6-13acba6ba7ee None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:34.315303 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-6b627486-c0aa-4e5c-95f6-13acba6ba7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:34.329527 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b627486-c0aa-4e5c-95f6-13acba6ba7ee None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:34.334673 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a7264-3436-4e7a-8060-24a6fb90d0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:34.335363 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a7264-3436-4e7a-8060-24a6fb90d0d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:34.336255 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a7264-3436-4e7a-8060-24a6fb90d0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:34.336613 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a7264-3436-4e7a-8060-24a6fb90d0d9 admin admin] PATH_INFO: `/v3/projects/8ab78d41be794a73a4bbaa7c0c934b85` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:34.338601 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a7264-3436-4e7a-8060-24a6fb90d0d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ab78d41be794a73a4bbaa7c0c934b85)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:34.340540 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a7264-3436-4e7a-8060-24a6fb90d0d9 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:34.391559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a04a7264-3436-4e7a-8060-24a6fb90d0d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ab78d41be794a73a4bbaa7c0c934b85', 'request_id': 'req-a04a7264-3436-4e7a-8060-24a6fb90d0d9'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:34.399298 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a04a7264-3436-4e7a-8060-24a6fb90d0d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ab78d41be794a73a4bbaa7c0c934b85', 'request_id': 'req-a04a7264-3436-4e7a-8060-24a6fb90d0d9'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:34.405698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a04a7264-3436-4e7a-8060-24a6fb90d0d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ab78d41be794a73a4bbaa7c0c934b85', 'request_id': 'req-a04a7264-3436-4e7a-8060-24a6fb90d0d9'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:34.420344 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a04a7264-3436-4e7a-8060-24a6fb90d0d9 admin admin] The token cache is being invalidate because project 8ab78d41be794a73a4bbaa7c0c934b85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:25:34.420630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a04a7264-3436-4e7a-8060-24a6fb90d0d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:34.421325 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-a04a7264-3436-4e7a-8060-24a6fb90d0d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:34.422870 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1420/2841] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:25:34 2022] DELETE /identity/v3/projects/8ab78d41be794a73a4bbaa7c0c934b85 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:34.427035 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871edcf-4adc-419f-aa17-e480663cb9be service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:34.437423 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871edcf-4adc-419f-aa17-e480663cb9be service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:34.457426 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:34.501650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cdd831-f102-405f-a7f3-87c52722cb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:34.502339 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdd831-f102-405f-a7f3-87c52722cb59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:34.502621 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdd831-f102-405f-a7f3-87c52722cb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:34.502895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdd831-f102-405f-a7f3-87c52722cb59 admin admin] PATH_INFO: `/v3/users/746130caf16647708c87b65a165e3207` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:34.504075 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cdd831-f102-405f-a7f3-87c52722cb59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=746130caf16647708c87b65a165e3207)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:34.506128 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cdd831-f102-405f-a7f3-87c52722cb59 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:34.559213 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-89cdd831-f102-405f-a7f3-87c52722cb59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '746130caf16647708c87b65a165e3207'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:34.572938 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-89cdd831-f102-405f-a7f3-87c52722cb59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '746130caf16647708c87b65a165e3207'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:34.580509 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-89cdd831-f102-405f-a7f3-87c52722cb59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '746130caf16647708c87b65a165e3207'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:34.587962 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-89cdd831-f102-405f-a7f3-87c52722cb59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '746130caf16647708c87b65a165e3207'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:34.588632 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-89cdd831-f102-405f-a7f3-87c52722cb59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '746130caf16647708c87b65a165e3207'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:34.594865 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1422/2842] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:25:34 2022] DELETE /identity/v3/users/746130caf16647708c87b65a165e3207 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:34.892133 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a7264-3436-4e7a-8060-24a6fb90d0d9 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:34.893829 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-a04a7264-3436-4e7a-8060-24a6fb90d0d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:34.911189 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a7264-3436-4e7a-8060-24a6fb90d0d9 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:34.974277 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd4533-4bc7-41a8-ad07-e87282a84b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:34.974940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd4533-4bc7-41a8-ad07-e87282a84b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:34.975221 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd4533-4bc7-41a8-ad07-e87282a84b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:34.975496 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd4533-4bc7-41a8-ad07-e87282a84b3b admin admin] PATH_INFO: `/v3/projects/3002ca775c0d4d8ea85c5e35751fcc19` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:34.977223 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd4533-4bc7-41a8-ad07-e87282a84b3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3002ca775c0d4d8ea85c5e35751fcc19)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:34.979310 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd4533-4bc7-41a8-ad07-e87282a84b3b admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:35.011432 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:35.031557 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-77bd4533-4bc7-41a8-ad07-e87282a84b3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3002ca775c0d4d8ea85c5e35751fcc19', 'request_id': 'req-77bd4533-4bc7-41a8-ad07-e87282a84b3b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:35.038062 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-77bd4533-4bc7-41a8-ad07-e87282a84b3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3002ca775c0d4d8ea85c5e35751fcc19', 'request_id': 'req-77bd4533-4bc7-41a8-ad07-e87282a84b3b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:35.044741 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-77bd4533-4bc7-41a8-ad07-e87282a84b3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3002ca775c0d4d8ea85c5e35751fcc19', 'request_id': 'req-77bd4533-4bc7-41a8-ad07-e87282a84b3b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:35.057635 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-77bd4533-4bc7-41a8-ad07-e87282a84b3b admin admin] The token cache is being invalidate because project 3002ca775c0d4d8ea85c5e35751fcc19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:25:35.057926 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-77bd4533-4bc7-41a8-ad07-e87282a84b3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:35.058568 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-77bd4533-4bc7-41a8-ad07-e87282a84b3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:35.060069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1421/2843] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:25:34 2022] DELETE /identity/v3/projects/3002ca775c0d4d8ea85c5e35751fcc19 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:35.065155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9de7c5-49cc-4bb7-b664-92a6735edd4a None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:35.065449 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9de7c5-49cc-4bb7-b664-92a6735edd4a None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:35.065760 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9de7c5-49cc-4bb7-b664-92a6735edd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:35.087136 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-0b9de7c5-49cc-4bb7-b664-92a6735edd4a None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:35.088539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-0b9de7c5-49cc-4bb7-b664-92a6735edd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:35.153995 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1423/2844] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:35.161883 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c4237-cced-47ab-b0bf-64ac77c0d9df None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:35.162102 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c4237-cced-47ab-b0bf-64ac77c0d9df None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:35.162402 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c4237-cced-47ab-b0bf-64ac77c0d9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:35.185417 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-c73c4237-cced-47ab-b0bf-64ac77c0d9df None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:35.186748 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c73c4237-cced-47ab-b0bf-64ac77c0d9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:35.195453 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1422/2845] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:35.199913 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9de7c5-49cc-4bb7-b664-92a6735edd4a None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:35.210401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9de7c5-49cc-4bb7-b664-92a6735edd4a None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:35.215488 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed17a5-c007-4ed8-974a-4ca238c8fdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:35.216114 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed17a5-c007-4ed8-974a-4ca238c8fdfb admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:35.216392 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed17a5-c007-4ed8-974a-4ca238c8fdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:35.216664 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed17a5-c007-4ed8-974a-4ca238c8fdfb admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:35.217319 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed17a5-c007-4ed8-974a-4ca238c8fdfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:25:35.217620 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed17a5-c007-4ed8-974a-4ca238c8fdfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:35.219267 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed17a5-c007-4ed8-974a-4ca238c8fdfb admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:35.232609 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1424/2846] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:25:35.237088 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c4237-cced-47ab-b0bf-64ac77c0d9df None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:35.247555 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c4237-cced-47ab-b0bf-64ac77c0d9df None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:35.252356 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701586f6-b78d-41f5-a1ee-9044ecc7d09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:35.252966 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701586f6-b78d-41f5-a1ee-9044ecc7d09c admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:35.253246 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701586f6-b78d-41f5-a1ee-9044ecc7d09c admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:35.253519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701586f6-b78d-41f5-a1ee-9044ecc7d09c admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:35.254208 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701586f6-b78d-41f5-a1ee-9044ecc7d09c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:35.256129 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701586f6-b78d-41f5-a1ee-9044ecc7d09c admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:35.284690 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1423/2847] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:25:35 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:25:35.289449 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed17a5-c007-4ed8-974a-4ca238c8fdfb admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:35.300432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed17a5-c007-4ed8-974a-4ca238c8fdfb admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:35.333553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:35.376485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0c1ed-58f9-4c0f-ac48-a9a3e4351e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:35.377606 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0c1ed-58f9-4c0f-ac48-a9a3e4351e42 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:35.377929 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0c1ed-58f9-4c0f-ac48-a9a3e4351e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:35.378194 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0c1ed-58f9-4c0f-ac48-a9a3e4351e42 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:35.378844 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad0c1ed-58f9-4c0f-ac48-a9a3e4351e42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:35.380986 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad0c1ed-58f9-4c0f-ac48-a9a3e4351e42 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:35.399317 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e61fd-5174-4550-8cf6-bf3cb32ddd78 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:35.400477 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e61fd-5174-4550-8cf6-bf3cb32ddd78 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:35.400477 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e61fd-5174-4550-8cf6-bf3cb32ddd78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:35.405574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1425/2848] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:25:35 2022] POST /identity/v3/users => generated 283 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:25:35.410147 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0c1ed-58f9-4c0f-ac48-a9a3e4351e42 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:35.421598 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0c1ed-58f9-4c0f-ac48-a9a3e4351e42 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:35.422306 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-3d5e61fd-5174-4550-8cf6-bf3cb32ddd78 None None] MFA Rules not processed for user `e0c76478b6694782b896cdf9cda32d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:35.423544 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-3d5e61fd-5174-4550-8cf6-bf3cb32ddd78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:35.427363 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00674e0-3f33-4faa-9bb8-86fa60185393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:35.428270 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00674e0-3f33-4faa-9bb8-86fa60185393 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:35.428559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00674e0-3f33-4faa-9bb8-86fa60185393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:35.428841 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00674e0-3f33-4faa-9bb8-86fa60185393 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:35.430295 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00674e0-3f33-4faa-9bb8-86fa60185393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:35.432007 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00674e0-3f33-4faa-9bb8-86fa60185393 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:35.445521 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1426/2849] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:25:35.449651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00674e0-3f33-4faa-9bb8-86fa60185393 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:35.462643 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00674e0-3f33-4faa-9bb8-86fa60185393 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:35.467582 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47fec1-0bd8-4529-a9fe-d1270d8521af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:35.468235 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47fec1-0bd8-4529-a9fe-d1270d8521af admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:35.468558 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47fec1-0bd8-4529-a9fe-d1270d8521af admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:35.468883 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47fec1-0bd8-4529-a9fe-d1270d8521af admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:35.469604 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47fec1-0bd8-4529-a9fe-d1270d8521af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:35.471232 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47fec1-0bd8-4529-a9fe-d1270d8521af admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:35.483834 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1427/2850] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:25:35.488052 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47fec1-0bd8-4529-a9fe-d1270d8521af admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:35.493529 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1424/2851] 10.222.0.73 () {58 vars in 967 bytes} [Mon Aug 29 20:25:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:25:35.499564 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47fec1-0bd8-4529-a9fe-d1270d8521af admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:35.505510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a6b18-fd4d-460f-b0d2-edec088d1b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:35.506247 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a6b18-fd4d-460f-b0d2-edec088d1b5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:35.506530 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a6b18-fd4d-460f-b0d2-edec088d1b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:35.506809 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a6b18-fd4d-460f-b0d2-edec088d1b5a admin admin] PATH_INFO: `/v3/projects/3085db4527174628bbda040c6630bc99/users/d3d0e1d6abcf418484f7b67ba3bd6441/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:35.525853 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a6b18-fd4d-460f-b0d2-edec088d1b5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3085db4527174628bbda040c6630bc99, user_id=d3d0e1d6abcf418484f7b67ba3bd6441, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:35.529147 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a6b18-fd4d-460f-b0d2-edec088d1b5a admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:35.537752 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e61fd-5174-4550-8cf6-bf3cb32ddd78 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:35.540103 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-3d5e61fd-5174-4550-8cf6-bf3cb32ddd78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:35.544166 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1428/2852] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:25:35 2022] PUT /identity/v3/projects/3085db4527174628bbda040c6630bc99/users/d3d0e1d6abcf418484f7b67ba3bd6441/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:35.612109 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35293d59-655f-4b03-a5d3-67fb6c9b3256 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:35.612399 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35293d59-655f-4b03-a5d3-67fb6c9b3256 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:35.612883 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35293d59-655f-4b03-a5d3-67fb6c9b3256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:35.635292 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e61fd-5174-4550-8cf6-bf3cb32ddd78 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:35.645146 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-35293d59-655f-4b03-a5d3-67fb6c9b3256 None None] MFA Rules not processed for user `d3d0e1d6abcf418484f7b67ba3bd6441`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:35.646603 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-35293d59-655f-4b03-a5d3-67fb6c9b3256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:35.660726 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:35.706602 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8164eb78-4a2a-47a0-820f-3747356c41d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92e30da8f394f3abc19b9e86ef126e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:35.707226 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164eb78-4a2a-47a0-820f-3747356c41d6 service swift] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:35.707508 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164eb78-4a2a-47a0-820f-3747356c41d6 service swift] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:35.707780 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164eb78-4a2a-47a0-820f-3747356c41d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:35.719431 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1429/2853] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:25:35.724015 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35293d59-655f-4b03-a5d3-67fb6c9b3256 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:35.770342 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8164eb78-4a2a-47a0-820f-3747356c41d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:35.771955 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8164eb78-4a2a-47a0-820f-3747356c41d6 service swift] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:35.780944 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1425/2854] 10.222.0.73 () {60 vars in 1384 bytes} [Mon Aug 29 20:25:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 20:25:35.793271 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35293d59-655f-4b03-a5d3-67fb6c9b3256 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:35.798834 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8c28a-e628-4555-81af-43bb3f248df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:35.799584 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8c28a-e628-4555-81af-43bb3f248df8 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:35.800023 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8c28a-e628-4555-81af-43bb3f248df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:35.800295 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8c28a-e628-4555-81af-43bb3f248df8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:35.800938 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8c28a-e628-4555-81af-43bb3f248df8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:35.802962 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8c28a-e628-4555-81af-43bb3f248df8 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:35.816379 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:35.852098 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1430/2855] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:25:35 2022] POST /identity/v3/projects => generated 365 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:25:35.855934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8164eb78-4a2a-47a0-820f-3747356c41d6 service swift] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:35.867352 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8164eb78-4a2a-47a0-820f-3747356c41d6 service swift] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:35.934968 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc253ce-98d2-40fa-8670-a2b74342e20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:35.935597 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc253ce-98d2-40fa-8670-a2b74342e20a admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:35.935870 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc253ce-98d2-40fa-8670-a2b74342e20a admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:35.936137 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc253ce-98d2-40fa-8670-a2b74342e20a admin admin] PATH_INFO: `/v3/users` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:35.936794 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc253ce-98d2-40fa-8670-a2b74342e20a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:35.938808 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc253ce-98d2-40fa-8670-a2b74342e20a admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:35.963282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1426/2856] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:25:35 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:25:35.967414 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8c28a-e628-4555-81af-43bb3f248df8 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:35.980407 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8c28a-e628-4555-81af-43bb3f248df8 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:35.993643 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccda9fb-233b-498a-b781-dcb68e83d1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:35.994314 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccda9fb-233b-498a-b781-dcb68e83d1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:35.994599 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccda9fb-233b-498a-b781-dcb68e83d1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:35.994878 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccda9fb-233b-498a-b781-dcb68e83d1b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:35.995546 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccda9fb-233b-498a-b781-dcb68e83d1b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:35.997235 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccda9fb-233b-498a-b781-dcb68e83d1b0 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:36.009753 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1431/2857] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:25:36.013298 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc253ce-98d2-40fa-8670-a2b74342e20a admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:36.023979 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc253ce-98d2-40fa-8670-a2b74342e20a admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:36.029630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862e1c1-f3bb-4ca8-9611-fc349feda958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:36.030353 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862e1c1-f3bb-4ca8-9611-fc349feda958 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:36.030626 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862e1c1-f3bb-4ca8-9611-fc349feda958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:36.030888 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862e1c1-f3bb-4ca8-9611-fc349feda958 admin admin] PATH_INFO: `/v3/projects/efc25ee3297e4ba79a74d6397cdc4692/users/17f2081159bb43f4bc2a2105de2cfdc3/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:36.048793 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7862e1c1-f3bb-4ca8-9611-fc349feda958 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc25ee3297e4ba79a74d6397cdc4692, user_id=17f2081159bb43f4bc2a2105de2cfdc3, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:36.051417 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7862e1c1-f3bb-4ca8-9611-fc349feda958 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:36.067678 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1427/2858] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:25:36 2022] PUT /identity/v3/projects/efc25ee3297e4ba79a74d6397cdc4692/users/17f2081159bb43f4bc2a2105de2cfdc3/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:36.071665 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccda9fb-233b-498a-b781-dcb68e83d1b0 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:36.083762 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccda9fb-233b-498a-b781-dcb68e83d1b0 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:36.160542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724d88d-d0e8-4a33-badb-a65479c14a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:36.161275 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724d88d-d0e8-4a33-badb-a65479c14a71 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:36.161648 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724d88d-d0e8-4a33-badb-a65479c14a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:36.161959 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724d88d-d0e8-4a33-badb-a65479c14a71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:36.162619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6724d88d-d0e8-4a33-badb-a65479c14a71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:36.164327 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6724d88d-d0e8-4a33-badb-a65479c14a71 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:36.177697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1432/2859] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:25:36.182666 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862e1c1-f3bb-4ca8-9611-fc349feda958 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:36.199404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862e1c1-f3bb-4ca8-9611-fc349feda958 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:36.205167 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6068a-ddac-4b47-b448-a1eb24dae7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:36.205903 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6068a-ddac-4b47-b448-a1eb24dae7a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:36.206180 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6068a-ddac-4b47-b448-a1eb24dae7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:36.206452 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6068a-ddac-4b47-b448-a1eb24dae7a0 admin admin] PATH_INFO: `/v3/projects/efc25ee3297e4ba79a74d6397cdc4692/users/17f2081159bb43f4bc2a2105de2cfdc3/roles/296ee2c194824485af71618619f82e09` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:36.208583 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc6068a-ddac-4b47-b448-a1eb24dae7a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc25ee3297e4ba79a74d6397cdc4692, user_id=17f2081159bb43f4bc2a2105de2cfdc3, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:36.211991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc6068a-ddac-4b47-b448-a1eb24dae7a0 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:36.224214 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1428/2860] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:25:36 2022] PUT /identity/v3/projects/efc25ee3297e4ba79a74d6397cdc4692/users/17f2081159bb43f4bc2a2105de2cfdc3/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:36.235277 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf53d7-2340-4426-acc9-1adb3ff909c6 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:36.235706 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf53d7-2340-4426-acc9-1adb3ff909c6 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:36.236198 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf53d7-2340-4426-acc9-1adb3ff909c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:36.288729 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-c4cf53d7-2340-4426-acc9-1adb3ff909c6 None None] MFA Rules not processed for user `17f2081159bb43f4bc2a2105de2cfdc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:36.290868 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c4cf53d7-2340-4426-acc9-1adb3ff909c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:36.397228 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1433/2861] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:25:36.411321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6068a-ddac-4b47-b448-a1eb24dae7a0 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:36.414043 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-ecc6068a-ddac-4b47-b448-a1eb24dae7a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:36.434468 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:36.485647 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6068a-ddac-4b47-b448-a1eb24dae7a0 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:36.489459 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52800b4e-6dc3-4ca6-ae19-8fd737fee82e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:36.490116 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52800b4e-6dc3-4ca6-ae19-8fd737fee82e service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:36.490392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52800b4e-6dc3-4ca6-ae19-8fd737fee82e service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:36.490696 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52800b4e-6dc3-4ca6-ae19-8fd737fee82e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:36.500440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52800b4e-6dc3-4ca6-ae19-8fd737fee82e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:36.501906 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52800b4e-6dc3-4ca6-ae19-8fd737fee82e service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:36.513126 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1429/2862] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:36 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:36.600728 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf53d7-2340-4426-acc9-1adb3ff909c6 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:36.611766 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf53d7-2340-4426-acc9-1adb3ff909c6 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:36.615326 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2b2e2-ced8-4596-ab81-a3b7a6541492 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:36.615943 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2b2e2-ced8-4596-ab81-a3b7a6541492 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:36.616216 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2b2e2-ced8-4596-ab81-a3b7a6541492 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:36.616527 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2b2e2-ced8-4596-ab81-a3b7a6541492 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:36.677973 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:36.681230 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2b2e2-ced8-4596-ab81-a3b7a6541492 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:36.683845 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2b2e2-ced8-4596-ab81-a3b7a6541492 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:36.695664 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1434/2863] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:43.241565 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52800b4e-6dc3-4ca6-ae19-8fd737fee82e service cinder] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:43.251174 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-52800b4e-6dc3-4ca6-ae19-8fd737fee82e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:43.365391 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52800b4e-6dc3-4ca6-ae19-8fd737fee82e service cinder] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:43.374949 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5de7a-a306-413c-8cd5-97c749f2cabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:43.375744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5de7a-a306-413c-8cd5-97c749f2cabf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:43.376140 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5de7a-a306-413c-8cd5-97c749f2cabf admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:43.376429 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5de7a-a306-413c-8cd5-97c749f2cabf admin admin] PATH_INFO: `/v3/users/fc11bab9d92c48e3938593035b837da1` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:43.377861 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5de7a-a306-413c-8cd5-97c749f2cabf admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc11bab9d92c48e3938593035b837da1)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:43.380657 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5de7a-a306-413c-8cd5-97c749f2cabf admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:43.466629 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-6ad5de7a-a306-413c-8cd5-97c749f2cabf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc11bab9d92c48e3938593035b837da1'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:43.481206 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-6ad5de7a-a306-413c-8cd5-97c749f2cabf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc11bab9d92c48e3938593035b837da1'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:43.505276 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-6ad5de7a-a306-413c-8cd5-97c749f2cabf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc11bab9d92c48e3938593035b837da1'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:43.514510 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-6ad5de7a-a306-413c-8cd5-97c749f2cabf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc11bab9d92c48e3938593035b837da1'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:43.515289 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-6ad5de7a-a306-413c-8cd5-97c749f2cabf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc11bab9d92c48e3938593035b837da1'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:43.534144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1430/2864] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:25:43 2022] DELETE /identity/v3/users/fc11bab9d92c48e3938593035b837da1 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:43.853874 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2b2e2-ced8-4596-ab81-a3b7a6541492 service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:43.874156 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2b2e2-ced8-4596-ab81-a3b7a6541492 service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:44.054913 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc397c8-2763-447c-a287-1d477f5ca1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:44.056312 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc397c8-2763-447c-a287-1d477f5ca1bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:44.056703 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc397c8-2763-447c-a287-1d477f5ca1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:44.057083 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc397c8-2763-447c-a287-1d477f5ca1bd admin admin] PATH_INFO: `/v3/projects/9f9f5a2386e2451f9a9b833859343595` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:44.058712 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc397c8-2763-447c-a287-1d477f5ca1bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f9f5a2386e2451f9a9b833859343595)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:44.072640 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc397c8-2763-447c-a287-1d477f5ca1bd admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:44.175645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-3cc397c8-2763-447c-a287-1d477f5ca1bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f9f5a2386e2451f9a9b833859343595', 'request_id': 'req-3cc397c8-2763-447c-a287-1d477f5ca1bd'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:44.185076 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-3cc397c8-2763-447c-a287-1d477f5ca1bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f9f5a2386e2451f9a9b833859343595', 'request_id': 'req-3cc397c8-2763-447c-a287-1d477f5ca1bd'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:44.198371 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-3cc397c8-2763-447c-a287-1d477f5ca1bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f9f5a2386e2451f9a9b833859343595', 'request_id': 'req-3cc397c8-2763-447c-a287-1d477f5ca1bd'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:44.215243 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-3cc397c8-2763-447c-a287-1d477f5ca1bd admin admin] The token cache is being invalidate because project 9f9f5a2386e2451f9a9b833859343595 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:25:44.215881 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-3cc397c8-2763-447c-a287-1d477f5ca1bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:44.216895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-3cc397c8-2763-447c-a287-1d477f5ca1bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:44.218838 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1435/2865] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:25:43 2022] DELETE /identity/v3/projects/9f9f5a2386e2451f9a9b833859343595 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:44.222774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5de7a-a306-413c-8cd5-97c749f2cabf admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:44.224428 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-6ad5de7a-a306-413c-8cd5-97c749f2cabf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:44.242848 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5de7a-a306-413c-8cd5-97c749f2cabf admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:44.298043 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:44.325609 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8b8c2-130e-40d6-91f9-d483bbe97487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:44.326307 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8b8c2-130e-40d6-91f9-d483bbe97487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:44.326559 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8b8c2-130e-40d6-91f9-d483bbe97487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:44.326829 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8b8c2-130e-40d6-91f9-d483bbe97487 admin admin] PATH_INFO: `/v3/users/1e501bd548764abe9c3ab9898f7a5d08` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:44.328028 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c8b8c2-130e-40d6-91f9-d483bbe97487 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e501bd548764abe9c3ab9898f7a5d08)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:44.330146 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c8b8c2-130e-40d6-91f9-d483bbe97487 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:44.389656 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-05c8b8c2-130e-40d6-91f9-d483bbe97487 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e501bd548764abe9c3ab9898f7a5d08'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:44.400329 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-05c8b8c2-130e-40d6-91f9-d483bbe97487 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e501bd548764abe9c3ab9898f7a5d08'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:44.417889 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-05c8b8c2-130e-40d6-91f9-d483bbe97487 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e501bd548764abe9c3ab9898f7a5d08'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:44.427258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-05c8b8c2-130e-40d6-91f9-d483bbe97487 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e501bd548764abe9c3ab9898f7a5d08'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:44.427991 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-05c8b8c2-130e-40d6-91f9-d483bbe97487 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e501bd548764abe9c3ab9898f7a5d08'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:44.434724 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1431/2866] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:25:44 2022] DELETE /identity/v3/users/1e501bd548764abe9c3ab9898f7a5d08 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:25:44.635659 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc397c8-2763-447c-a287-1d477f5ca1bd admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:44.637449 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-3cc397c8-2763-447c-a287-1d477f5ca1bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:44.651651 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc397c8-2763-447c-a287-1d477f5ca1bd admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:44.742132 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba2556-7c81-4403-af91-afcd454fbadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:44.742909 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba2556-7c81-4403-af91-afcd454fbadb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:44.743262 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba2556-7c81-4403-af91-afcd454fbadb admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:44.743608 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba2556-7c81-4403-af91-afcd454fbadb admin admin] PATH_INFO: `/v3/projects/79c093bab922455e9fc9bbf0b93fa278` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:44.744853 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba2556-7c81-4403-af91-afcd454fbadb admin admin] RBAC: Authorizing `identity:delete_project(project_id=79c093bab922455e9fc9bbf0b93fa278)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:44.746935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba2556-7c81-4403-af91-afcd454fbadb admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:44.788199 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:44.818670 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-94ba2556-7c81-4403-af91-afcd454fbadb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79c093bab922455e9fc9bbf0b93fa278', 'request_id': 'req-94ba2556-7c81-4403-af91-afcd454fbadb'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:44.825426 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-94ba2556-7c81-4403-af91-afcd454fbadb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79c093bab922455e9fc9bbf0b93fa278', 'request_id': 'req-94ba2556-7c81-4403-af91-afcd454fbadb'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:44.843057 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-94ba2556-7c81-4403-af91-afcd454fbadb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79c093bab922455e9fc9bbf0b93fa278', 'request_id': 'req-94ba2556-7c81-4403-af91-afcd454fbadb'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:44.859646 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-94ba2556-7c81-4403-af91-afcd454fbadb admin admin] The token cache is being invalidate because project 79c093bab922455e9fc9bbf0b93fa278 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:25:44.860974 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-94ba2556-7c81-4403-af91-afcd454fbadb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:44.860974 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-94ba2556-7c81-4403-af91-afcd454fbadb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:44.862529 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1436/2867] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:25:44 2022] DELETE /identity/v3/projects/79c093bab922455e9fc9bbf0b93fa278 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:46.241005 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8b8c2-130e-40d6-91f9-d483bbe97487 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:46.244255 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-05c8b8c2-130e-40d6-91f9-d483bbe97487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:46.335125 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8b8c2-130e-40d6-91f9-d483bbe97487 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:46.452912 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:46.467615 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58906831-c564-4939-ad93-d5729b1e55b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab2e318b2854c46bc7ed2151a44a2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:46.468710 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58906831-c564-4939-ad93-d5729b1e55b8 service placement] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:46.469698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58906831-c564-4939-ad93-d5729b1e55b8 service placement] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:46.469939 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58906831-c564-4939-ad93-d5729b1e55b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:46.474258 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-58906831-c564-4939-ad93-d5729b1e55b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:46.495031 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58906831-c564-4939-ad93-d5729b1e55b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:46.496723 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58906831-c564-4939-ad93-d5729b1e55b8 service placement] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:46.521420 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1432/2868] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:47.985863 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba2556-7c81-4403-af91-afcd454fbadb admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:47.987570 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-94ba2556-7c81-4403-af91-afcd454fbadb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:48.079178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba2556-7c81-4403-af91-afcd454fbadb admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:48.087495 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ca390-937c-4bb0-82e2-9c6f29a2678a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:48.088165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ca390-937c-4bb0-82e2-9c6f29a2678a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:48.088453 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ca390-937c-4bb0-82e2-9c6f29a2678a admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:48.088724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ca390-937c-4bb0-82e2-9c6f29a2678a admin admin] PATH_INFO: `/v3/users/d3d0e1d6abcf418484f7b67ba3bd6441` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:48.089962 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ca390-937c-4bb0-82e2-9c6f29a2678a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3d0e1d6abcf418484f7b67ba3bd6441)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:48.092098 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ca390-937c-4bb0-82e2-9c6f29a2678a admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:48.152367 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a51ca390-937c-4bb0-82e2-9c6f29a2678a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3d0e1d6abcf418484f7b67ba3bd6441'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:48.168158 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a51ca390-937c-4bb0-82e2-9c6f29a2678a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3d0e1d6abcf418484f7b67ba3bd6441'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:48.177630 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a51ca390-937c-4bb0-82e2-9c6f29a2678a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3d0e1d6abcf418484f7b67ba3bd6441'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:48.185786 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a51ca390-937c-4bb0-82e2-9c6f29a2678a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3d0e1d6abcf418484f7b67ba3bd6441'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:48.188486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a51ca390-937c-4bb0-82e2-9c6f29a2678a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3d0e1d6abcf418484f7b67ba3bd6441'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:48.192959 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1437/2869] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:25:47 2022] DELETE /identity/v3/users/d3d0e1d6abcf418484f7b67ba3bd6441 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:48.198168 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8fd0e-0da1-4c45-8562-3b80201e2d26 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:48.198869 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8fd0e-0da1-4c45-8562-3b80201e2d26 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:48.198869 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8fd0e-0da1-4c45-8562-3b80201e2d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:48.226700 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-b1a8fd0e-0da1-4c45-8562-3b80201e2d26 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:48.227940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-b1a8fd0e-0da1-4c45-8562-3b80201e2d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:48.305313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1433/2870] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:25:48.315372 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ca390-937c-4bb0-82e2-9c6f29a2678a admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:48.316990 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-a51ca390-937c-4bb0-82e2-9c6f29a2678a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:48.364701 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:48.384060 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ca390-937c-4bb0-82e2-9c6f29a2678a admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:48.388602 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e54687-ccc9-4f19-9566-3bb8692df660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:48.389252 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e54687-ccc9-4f19-9566-3bb8692df660 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:48.389537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e54687-ccc9-4f19-9566-3bb8692df660 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:48.389860 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e54687-ccc9-4f19-9566-3bb8692df660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:48.401654 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e54687-ccc9-4f19-9566-3bb8692df660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:48.403220 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e54687-ccc9-4f19-9566-3bb8692df660 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:48.415234 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1438/2871] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:48.708084 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8fd0e-0da1-4c45-8562-3b80201e2d26 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:48.709636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-b1a8fd0e-0da1-4c45-8562-3b80201e2d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:48.722475 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8fd0e-0da1-4c45-8562-3b80201e2d26 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:48.728153 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7ebec-2d72-4104-96ca-a372a79a274e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:48.728810 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7ebec-2d72-4104-96ca-a372a79a274e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:48.729092 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7ebec-2d72-4104-96ca-a372a79a274e admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:48.729371 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7ebec-2d72-4104-96ca-a372a79a274e admin admin] PATH_INFO: `/v3/projects/3085db4527174628bbda040c6630bc99` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:48.730733 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7ebec-2d72-4104-96ca-a372a79a274e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3085db4527174628bbda040c6630bc99)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:48.732776 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7ebec-2d72-4104-96ca-a372a79a274e admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:48.793033 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c8c7ebec-2d72-4104-96ca-a372a79a274e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3085db4527174628bbda040c6630bc99', 'request_id': 'req-c8c7ebec-2d72-4104-96ca-a372a79a274e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:48.799800 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c8c7ebec-2d72-4104-96ca-a372a79a274e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3085db4527174628bbda040c6630bc99', 'request_id': 'req-c8c7ebec-2d72-4104-96ca-a372a79a274e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:48.806489 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c8c7ebec-2d72-4104-96ca-a372a79a274e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3085db4527174628bbda040c6630bc99', 'request_id': 'req-c8c7ebec-2d72-4104-96ca-a372a79a274e'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:48.820611 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c8c7ebec-2d72-4104-96ca-a372a79a274e admin admin] The token cache is being invalidate because project 3085db4527174628bbda040c6630bc99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:25:48.820905 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c8c7ebec-2d72-4104-96ca-a372a79a274e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:48.821569 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-c8c7ebec-2d72-4104-96ca-a372a79a274e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:48.823123 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1434/2872] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:25:48 2022] DELETE /identity/v3/projects/3085db4527174628bbda040c6630bc99 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:48.827381 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e54687-ccc9-4f19-9566-3bb8692df660 service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:48.838741 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e54687-ccc9-4f19-9566-3bb8692df660 service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:48.906417 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98fe1f4-0581-46d7-bb8e-ad4efdc85273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:48.907069 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fe1f4-0581-46d7-bb8e-ad4efdc85273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:48.907365 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fe1f4-0581-46d7-bb8e-ad4efdc85273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:48.907624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fe1f4-0581-46d7-bb8e-ad4efdc85273 admin admin] PATH_INFO: `/v3/users/17f2081159bb43f4bc2a2105de2cfdc3` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:48.908831 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98fe1f4-0581-46d7-bb8e-ad4efdc85273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17f2081159bb43f4bc2a2105de2cfdc3)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:48.910824 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98fe1f4-0581-46d7-bb8e-ad4efdc85273 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:48.958981 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-c98fe1f4-0581-46d7-bb8e-ad4efdc85273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17f2081159bb43f4bc2a2105de2cfdc3'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:48.971650 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-c98fe1f4-0581-46d7-bb8e-ad4efdc85273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17f2081159bb43f4bc2a2105de2cfdc3'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:48.979919 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-c98fe1f4-0581-46d7-bb8e-ad4efdc85273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17f2081159bb43f4bc2a2105de2cfdc3'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:48.988537 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-c98fe1f4-0581-46d7-bb8e-ad4efdc85273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17f2081159bb43f4bc2a2105de2cfdc3'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:48.989309 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-c98fe1f4-0581-46d7-bb8e-ad4efdc85273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17f2081159bb43f4bc2a2105de2cfdc3'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:48.997316 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1439/2873] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:25:48 2022] DELETE /identity/v3/users/17f2081159bb43f4bc2a2105de2cfdc3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:49.321404 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7ebec-2d72-4104-96ca-a372a79a274e admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:49.323001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c8c7ebec-2d72-4104-96ca-a372a79a274e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:49.338712 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7ebec-2d72-4104-96ca-a372a79a274e admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:49.427688 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5152f7-f8b8-4e96-8be8-5e586a2d2095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:49.429424 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5152f7-f8b8-4e96-8be8-5e586a2d2095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:49.429833 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5152f7-f8b8-4e96-8be8-5e586a2d2095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:49.430261 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5152f7-f8b8-4e96-8be8-5e586a2d2095 admin admin] PATH_INFO: `/v3/projects/efc25ee3297e4ba79a74d6397cdc4692` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:49.432025 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5152f7-f8b8-4e96-8be8-5e586a2d2095 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efc25ee3297e4ba79a74d6397cdc4692)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:49.434477 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5152f7-f8b8-4e96-8be8-5e586a2d2095 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:49.454764 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:49.506281 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-1e5152f7-f8b8-4e96-8be8-5e586a2d2095 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efc25ee3297e4ba79a74d6397cdc4692', 'request_id': 'req-1e5152f7-f8b8-4e96-8be8-5e586a2d2095'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:49.513731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-1e5152f7-f8b8-4e96-8be8-5e586a2d2095 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efc25ee3297e4ba79a74d6397cdc4692', 'request_id': 'req-1e5152f7-f8b8-4e96-8be8-5e586a2d2095'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:49.519570 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-1e5152f7-f8b8-4e96-8be8-5e586a2d2095 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efc25ee3297e4ba79a74d6397cdc4692', 'request_id': 'req-1e5152f7-f8b8-4e96-8be8-5e586a2d2095'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:49.534268 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-1e5152f7-f8b8-4e96-8be8-5e586a2d2095 admin admin] The token cache is being invalidate because project efc25ee3297e4ba79a74d6397cdc4692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:25:49.534583 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-1e5152f7-f8b8-4e96-8be8-5e586a2d2095 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:49.535230 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-1e5152f7-f8b8-4e96-8be8-5e586a2d2095 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:49.536758 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1435/2874] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:25:49 2022] DELETE /identity/v3/projects/efc25ee3297e4ba79a74d6397cdc4692 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:25:49.543591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020413e-ea6f-4db6-9b48-798a3bb04ccd None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:49.543878 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020413e-ea6f-4db6-9b48-798a3bb04ccd None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:49.544159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020413e-ea6f-4db6-9b48-798a3bb04ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:49.567243 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-d020413e-ea6f-4db6-9b48-798a3bb04ccd None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:49.569263 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-d020413e-ea6f-4db6-9b48-798a3bb04ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:49.650031 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:49.665071 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1440/2875] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:49.674866 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6fc7a-b590-4f9c-a9de-60831d4982b6 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:49.675410 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6fc7a-b590-4f9c-a9de-60831d4982b6 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:49.675867 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6fc7a-b590-4f9c-a9de-60831d4982b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:49.701882 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-9dc6fc7a-b590-4f9c-a9de-60831d4982b6 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:49.703219 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9dc6fc7a-b590-4f9c-a9de-60831d4982b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:49.713096 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1436/2876] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:49.717774 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020413e-ea6f-4db6-9b48-798a3bb04ccd None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:49.728645 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020413e-ea6f-4db6-9b48-798a3bb04ccd None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:49.733566 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad235c30-23a8-4756-9d41-6549f520dfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:49.734240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad235c30-23a8-4756-9d41-6549f520dfba admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:49.734682 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad235c30-23a8-4756-9d41-6549f520dfba admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:49.735174 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad235c30-23a8-4756-9d41-6549f520dfba admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:49.735863 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad235c30-23a8-4756-9d41-6549f520dfba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:25:49.736164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad235c30-23a8-4756-9d41-6549f520dfba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:49.737921 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad235c30-23a8-4756-9d41-6549f520dfba admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:49.754411 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1441/2877] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:25:49.760941 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6fc7a-b590-4f9c-a9de-60831d4982b6 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:49.771297 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6fc7a-b590-4f9c-a9de-60831d4982b6 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:49.778030 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85e1ec-bad8-4c8a-8d50-a037e1bcd14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:49.778805 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85e1ec-bad8-4c8a-8d50-a037e1bcd14a admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:49.779089 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85e1ec-bad8-4c8a-8d50-a037e1bcd14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:49.779362 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85e1ec-bad8-4c8a-8d50-a037e1bcd14a admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:49.780374 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85e1ec-bad8-4c8a-8d50-a037e1bcd14a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:49.782202 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85e1ec-bad8-4c8a-8d50-a037e1bcd14a admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:49.828744 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1437/2878] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:25:49 2022] POST /identity/v3/projects => generated 369 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:25:49.830147 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad235c30-23a8-4756-9d41-6549f520dfba admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:49.842306 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad235c30-23a8-4756-9d41-6549f520dfba admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:49.924481 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a854477-1030-4947-ad73-5e11c8721c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:49.924481 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a854477-1030-4947-ad73-5e11c8721c16 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:49.924481 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a854477-1030-4947-ad73-5e11c8721c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:49.924481 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a854477-1030-4947-ad73-5e11c8721c16 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:49.925001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a854477-1030-4947-ad73-5e11c8721c16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:49.925001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a854477-1030-4947-ad73-5e11c8721c16 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:49.950675 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1442/2879] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:25:49 2022] POST /identity/v3/users => generated 285 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:25:49.954576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85e1ec-bad8-4c8a-8d50-a037e1bcd14a admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:49.968194 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85e1ec-bad8-4c8a-8d50-a037e1bcd14a admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:49.972961 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d30e1ee-ced3-4912-9786-8a5a5555944c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:49.973576 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30e1ee-ced3-4912-9786-8a5a5555944c admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:49.974032 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30e1ee-ced3-4912-9786-8a5a5555944c admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:49.974308 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30e1ee-ced3-4912-9786-8a5a5555944c admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:49.975905 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d30e1ee-ced3-4912-9786-8a5a5555944c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:49.977818 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d30e1ee-ced3-4912-9786-8a5a5555944c admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:49.988636 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1438/2880] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:25:49.994462 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a854477-1030-4947-ad73-5e11c8721c16 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:50.007574 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a854477-1030-4947-ad73-5e11c8721c16 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:50.013068 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b197f522-d9e7-4a4f-b592-58243972be07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:50.013924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197f522-d9e7-4a4f-b592-58243972be07 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:50.014303 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197f522-d9e7-4a4f-b592-58243972be07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:50.014665 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197f522-d9e7-4a4f-b592-58243972be07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:50.015627 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b197f522-d9e7-4a4f-b592-58243972be07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:50.017556 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b197f522-d9e7-4a4f-b592-58243972be07 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:50.030467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1443/2881] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:25:50.035071 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d30e1ee-ced3-4912-9786-8a5a5555944c admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:50.047940 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d30e1ee-ced3-4912-9786-8a5a5555944c admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:50.053687 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e845a6-e13c-4e54-9c33-2222e8f22baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:50.054392 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e845a6-e13c-4e54-9c33-2222e8f22baf admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:50.054675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e845a6-e13c-4e54-9c33-2222e8f22baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:50.054945 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e845a6-e13c-4e54-9c33-2222e8f22baf admin admin] PATH_INFO: `/v3/projects/1b703a4e83e74a099a8f2209f810ac0e/users/7c0a358eb3f448f0bec38b74b6b55601/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:50.071065 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e845a6-e13c-4e54-9c33-2222e8f22baf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b703a4e83e74a099a8f2209f810ac0e, user_id=7c0a358eb3f448f0bec38b74b6b55601, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:50.073803 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e845a6-e13c-4e54-9c33-2222e8f22baf admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:50.096616 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1439/2882] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:25:50 2022] PUT /identity/v3/projects/1b703a4e83e74a099a8f2209f810ac0e/users/7c0a358eb3f448f0bec38b74b6b55601/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:50.109736 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6114086-6917-46b4-bc04-96182433195a None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:50.110021 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6114086-6917-46b4-bc04-96182433195a None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:50.110341 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6114086-6917-46b4-bc04-96182433195a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:50.140854 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-c6114086-6917-46b4-bc04-96182433195a None None] MFA Rules not processed for user `7c0a358eb3f448f0bec38b74b6b55601`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:50.142161 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-c6114086-6917-46b4-bc04-96182433195a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:50.209486 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1444/2883] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:50.214328 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e845a6-e13c-4e54-9c33-2222e8f22baf admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:50.281272 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e845a6-e13c-4e54-9c33-2222e8f22baf admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:50.286277 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0516d8-7f9d-41cd-bf95-f768c06e73c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:50.286901 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0516d8-7f9d-41cd-bf95-f768c06e73c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:50.287177 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0516d8-7f9d-41cd-bf95-f768c06e73c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:50.287448 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0516d8-7f9d-41cd-bf95-f768c06e73c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:50.288117 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0516d8-7f9d-41cd-bf95-f768c06e73c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:50.290307 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0516d8-7f9d-41cd-bf95-f768c06e73c7 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:50.319025 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1440/2884] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:25:50 2022] POST /identity/v3/projects => generated 369 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:25:50.323689 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6114086-6917-46b4-bc04-96182433195a None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:50.336278 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6114086-6917-46b4-bc04-96182433195a None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:50.397924 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:50.410133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d27d2-f88d-4bf6-8623-3ed49059498b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:50.410836 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d27d2-f88d-4bf6-8623-3ed49059498b admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:50.411165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d27d2-f88d-4bf6-8623-3ed49059498b admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:50.411544 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d27d2-f88d-4bf6-8623-3ed49059498b admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:50.414869 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d27d2-f88d-4bf6-8623-3ed49059498b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:50.416893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d27d2-f88d-4bf6-8623-3ed49059498b admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:50.439718 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1445/2885] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:25:50 2022] POST /identity/v3/users => generated 291 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:25:50.443945 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0516d8-7f9d-41cd-bf95-f768c06e73c7 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:50.456284 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0516d8-7f9d-41cd-bf95-f768c06e73c7 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:50.461318 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a7c91-561b-4eec-9c48-5749b38388f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:50.461986 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a7c91-561b-4eec-9c48-5749b38388f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:50.462274 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a7c91-561b-4eec-9c48-5749b38388f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:50.462547 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a7c91-561b-4eec-9c48-5749b38388f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:50.463202 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a7c91-561b-4eec-9c48-5749b38388f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:50.464813 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a7c91-561b-4eec-9c48-5749b38388f3 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:50.477064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1441/2886] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:25:50.481485 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d27d2-f88d-4bf6-8623-3ed49059498b admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:50.493385 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d27d2-f88d-4bf6-8623-3ed49059498b admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:50.498632 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4eba3d-792a-4108-a2ce-8cae6644e3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:50.499319 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4eba3d-792a-4108-a2ce-8cae6644e3c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:50.499596 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4eba3d-792a-4108-a2ce-8cae6644e3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:50.499867 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4eba3d-792a-4108-a2ce-8cae6644e3c5 admin admin] PATH_INFO: `/v3/projects/bff1175c6a224f239ac05b25cbe9e48a/users/45e968332e8b4c7ba82af3349cd7a43e/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:50.517284 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4eba3d-792a-4108-a2ce-8cae6644e3c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bff1175c6a224f239ac05b25cbe9e48a, user_id=45e968332e8b4c7ba82af3349cd7a43e, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:50.520060 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4eba3d-792a-4108-a2ce-8cae6644e3c5 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:50.536146 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1446/2887] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:25:50 2022] PUT /identity/v3/projects/bff1175c6a224f239ac05b25cbe9e48a/users/45e968332e8b4c7ba82af3349cd7a43e/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:50.541013 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a7c91-561b-4eec-9c48-5749b38388f3 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:50.551664 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a7c91-561b-4eec-9c48-5749b38388f3 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:50.562699 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:50.652379 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74752092-52d6-4423-926e-0e1dfbc38cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:50.653141 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74752092-52d6-4423-926e-0e1dfbc38cfb admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:50.653462 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74752092-52d6-4423-926e-0e1dfbc38cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:50.653835 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74752092-52d6-4423-926e-0e1dfbc38cfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:50.654541 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74752092-52d6-4423-926e-0e1dfbc38cfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:50.656195 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74752092-52d6-4423-926e-0e1dfbc38cfb admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:50.668295 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1442/2888] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:25:50.673881 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4eba3d-792a-4108-a2ce-8cae6644e3c5 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:50.688945 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4eba3d-792a-4108-a2ce-8cae6644e3c5 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:50.693983 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb39c5-7480-425f-8d86-1733ee9ad0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:50.694827 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb39c5-7480-425f-8d86-1733ee9ad0f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:50.695097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb39c5-7480-425f-8d86-1733ee9ad0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:50.695399 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb39c5-7480-425f-8d86-1733ee9ad0f9 admin admin] PATH_INFO: `/v3/projects/bff1175c6a224f239ac05b25cbe9e48a/users/45e968332e8b4c7ba82af3349cd7a43e/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:50.697465 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb39c5-7480-425f-8d86-1733ee9ad0f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bff1175c6a224f239ac05b25cbe9e48a, user_id=45e968332e8b4c7ba82af3349cd7a43e, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:50.700135 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb39c5-7480-425f-8d86-1733ee9ad0f9 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:50.712244 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1447/2889] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:25:50 2022] PUT /identity/v3/projects/bff1175c6a224f239ac05b25cbe9e48a/users/45e968332e8b4c7ba82af3349cd7a43e/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:25:50.723227 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956523c7-41aa-4bc8-a001-ed850fa7f367 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:50.723577 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956523c7-41aa-4bc8-a001-ed850fa7f367 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:50.723957 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956523c7-41aa-4bc8-a001-ed850fa7f367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:50.756994 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-956523c7-41aa-4bc8-a001-ed850fa7f367 None None] MFA Rules not processed for user `45e968332e8b4c7ba82af3349cd7a43e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:50.758257 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-956523c7-41aa-4bc8-a001-ed850fa7f367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:50.840064 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1443/2890] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:50.854723 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb39c5-7480-425f-8d86-1733ee9ad0f9 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:50.856672 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-e3cb39c5-7480-425f-8d86-1733ee9ad0f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:50.926226 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb39c5-7480-425f-8d86-1733ee9ad0f9 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:50.930724 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbbd472-e98c-4cdb-8909-ac944afaa53f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:50.931343 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbd472-e98c-4cdb-8909-ac944afaa53f service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:50.931633 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbd472-e98c-4cdb-8909-ac944afaa53f service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:50.931942 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbd472-e98c-4cdb-8909-ac944afaa53f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:50.943050 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbbd472-e98c-4cdb-8909-ac944afaa53f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:50.945349 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbbd472-e98c-4cdb-8909-ac944afaa53f service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:50.967365 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1448/2891] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:50 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:51.025596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956523c7-41aa-4bc8-a001-ed850fa7f367 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:51.028453 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-956523c7-41aa-4bc8-a001-ed850fa7f367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:51.042123 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956523c7-41aa-4bc8-a001-ed850fa7f367 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:51.045934 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633f1e5-87e0-4dcb-af35-47d5253f3b50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:51.046519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633f1e5-87e0-4dcb-af35-47d5253f3b50 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:51.046848 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633f1e5-87e0-4dcb-af35-47d5253f3b50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:51.047040 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633f1e5-87e0-4dcb-af35-47d5253f3b50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:51.111470 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633f1e5-87e0-4dcb-af35-47d5253f3b50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:51.113002 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633f1e5-87e0-4dcb-af35-47d5253f3b50 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:51.123307 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1444/2892] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:54.722212 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbbd472-e98c-4cdb-8909-ac944afaa53f service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:54.791750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbbd472-e98c-4cdb-8909-ac944afaa53f service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:54.796624 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e1185-cff9-40db-b5c5-5a4206ae8afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:54.797274 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e1185-cff9-40db-b5c5-5a4206ae8afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:54.797551 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e1185-cff9-40db-b5c5-5a4206ae8afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:54.797854 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e1185-cff9-40db-b5c5-5a4206ae8afc admin admin] PATH_INFO: `/v3/users/7c0a358eb3f448f0bec38b74b6b55601` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:54.799038 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e1185-cff9-40db-b5c5-5a4206ae8afc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c0a358eb3f448f0bec38b74b6b55601)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:54.801171 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e1185-cff9-40db-b5c5-5a4206ae8afc admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:54.854715 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e14e1185-cff9-40db-b5c5-5a4206ae8afc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c0a358eb3f448f0bec38b74b6b55601'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:54.856893 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:54.871829 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e14e1185-cff9-40db-b5c5-5a4206ae8afc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c0a358eb3f448f0bec38b74b6b55601'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:54.879443 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e14e1185-cff9-40db-b5c5-5a4206ae8afc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c0a358eb3f448f0bec38b74b6b55601'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:54.887545 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e14e1185-cff9-40db-b5c5-5a4206ae8afc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c0a358eb3f448f0bec38b74b6b55601'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:54.888344 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e14e1185-cff9-40db-b5c5-5a4206ae8afc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c0a358eb3f448f0bec38b74b6b55601'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:54.894401 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1449/2893] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:25:54 2022] DELETE /identity/v3/users/7c0a358eb3f448f0bec38b74b6b55601 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:54.900058 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf317f-3c3d-4f9e-976d-a68c6eaaba20 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:54.900599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf317f-3c3d-4f9e-976d-a68c6eaaba20 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:54.900599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf317f-3c3d-4f9e-976d-a68c6eaaba20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:54.922605 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-2ebf317f-3c3d-4f9e-976d-a68c6eaaba20 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:54.923842 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-2ebf317f-3c3d-4f9e-976d-a68c6eaaba20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:54.954874 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=100 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:54.987767 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1445/2894] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:25:54.998794 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e1185-cff9-40db-b5c5-5a4206ae8afc admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:55.000240 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-e14e1185-cff9-40db-b5c5-5a4206ae8afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:55.061524 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e1185-cff9-40db-b5c5-5a4206ae8afc admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:55.065173 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b16a25-1f10-458e-add9-35b0b63a67d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:55.065832 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b16a25-1f10-458e-add9-35b0b63a67d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:55.066110 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b16a25-1f10-458e-add9-35b0b63a67d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:55.066419 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b16a25-1f10-458e-add9-35b0b63a67d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:55.075999 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b16a25-1f10-458e-add9-35b0b63a67d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:55.077974 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b16a25-1f10-458e-add9-35b0b63a67d1 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:55.088104 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1450/2895] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:25:55.446252 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf317f-3c3d-4f9e-976d-a68c6eaaba20 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:55.447731 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-2ebf317f-3c3d-4f9e-976d-a68c6eaaba20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:55.463082 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf317f-3c3d-4f9e-976d-a68c6eaaba20 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:55.468383 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2b42d-996a-4267-97af-b75a10015161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:55.469042 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2b42d-996a-4267-97af-b75a10015161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:55.469315 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2b42d-996a-4267-97af-b75a10015161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:55.469586 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2b42d-996a-4267-97af-b75a10015161 admin admin] PATH_INFO: `/v3/projects/1b703a4e83e74a099a8f2209f810ac0e` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:55.470810 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd2b42d-996a-4267-97af-b75a10015161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b703a4e83e74a099a8f2209f810ac0e)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:55.472761 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd2b42d-996a-4267-97af-b75a10015161 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:55.546401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-2fd2b42d-996a-4267-97af-b75a10015161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b703a4e83e74a099a8f2209f810ac0e', 'request_id': 'req-2fd2b42d-996a-4267-97af-b75a10015161'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:55.554282 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-2fd2b42d-996a-4267-97af-b75a10015161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b703a4e83e74a099a8f2209f810ac0e', 'request_id': 'req-2fd2b42d-996a-4267-97af-b75a10015161'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:55.561351 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-2fd2b42d-996a-4267-97af-b75a10015161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b703a4e83e74a099a8f2209f810ac0e', 'request_id': 'req-2fd2b42d-996a-4267-97af-b75a10015161'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:55.577238 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-2fd2b42d-996a-4267-97af-b75a10015161 admin admin] The token cache is being invalidate because project 1b703a4e83e74a099a8f2209f810ac0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:25:55.577534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-2fd2b42d-996a-4267-97af-b75a10015161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:55.578334 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-2fd2b42d-996a-4267-97af-b75a10015161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:55.580205 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1446/2896] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:25:55 2022] DELETE /identity/v3/projects/1b703a4e83e74a099a8f2209f810ac0e => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:25:55.584933 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b16a25-1f10-458e-add9-35b0b63a67d1 service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:55.595945 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b16a25-1f10-458e-add9-35b0b63a67d1 service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:55.667808 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200e47a-efa4-4364-b283-486d9d3cfc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:55.668492 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200e47a-efa4-4364-b283-486d9d3cfc33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:55.668774 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200e47a-efa4-4364-b283-486d9d3cfc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:55.669127 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200e47a-efa4-4364-b283-486d9d3cfc33 admin admin] PATH_INFO: `/v3/users/45e968332e8b4c7ba82af3349cd7a43e` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:55.670452 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8200e47a-efa4-4364-b283-486d9d3cfc33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45e968332e8b4c7ba82af3349cd7a43e)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:55.672479 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8200e47a-efa4-4364-b283-486d9d3cfc33 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:55.729895 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8200e47a-efa4-4364-b283-486d9d3cfc33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45e968332e8b4c7ba82af3349cd7a43e'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:55.743165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8200e47a-efa4-4364-b283-486d9d3cfc33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45e968332e8b4c7ba82af3349cd7a43e'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:55.753520 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8200e47a-efa4-4364-b283-486d9d3cfc33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45e968332e8b4c7ba82af3349cd7a43e'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:55.755084 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:55.770188 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8200e47a-efa4-4364-b283-486d9d3cfc33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45e968332e8b4c7ba82af3349cd7a43e'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:55.771476 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-8200e47a-efa4-4364-b283-486d9d3cfc33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45e968332e8b4c7ba82af3349cd7a43e'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:55.779246 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1451/2897] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:25:55 2022] DELETE /identity/v3/users/45e968332e8b4c7ba82af3349cd7a43e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:56.091788 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2b42d-996a-4267-97af-b75a10015161 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:56.093595 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-2fd2b42d-996a-4267-97af-b75a10015161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:56.127172 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2b42d-996a-4267-97af-b75a10015161 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:56.189394 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ce705-a02e-4ad2-be45-8b1d752554a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:56.190086 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ce705-a02e-4ad2-be45-8b1d752554a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:56.190370 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ce705-a02e-4ad2-be45-8b1d752554a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:56.190665 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ce705-a02e-4ad2-be45-8b1d752554a2 admin admin] PATH_INFO: `/v3/projects/bff1175c6a224f239ac05b25cbe9e48a` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:56.191839 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545ce705-a02e-4ad2-be45-8b1d752554a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bff1175c6a224f239ac05b25cbe9e48a)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:56.194029 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545ce705-a02e-4ad2-be45-8b1d752554a2 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:56.245020 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-545ce705-a02e-4ad2-be45-8b1d752554a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bff1175c6a224f239ac05b25cbe9e48a', 'request_id': 'req-545ce705-a02e-4ad2-be45-8b1d752554a2'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:56.251183 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-545ce705-a02e-4ad2-be45-8b1d752554a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bff1175c6a224f239ac05b25cbe9e48a', 'request_id': 'req-545ce705-a02e-4ad2-be45-8b1d752554a2'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:56.255010 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:56.257859 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-545ce705-a02e-4ad2-be45-8b1d752554a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bff1175c6a224f239ac05b25cbe9e48a', 'request_id': 'req-545ce705-a02e-4ad2-be45-8b1d752554a2'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:56.271615 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-545ce705-a02e-4ad2-be45-8b1d752554a2 admin admin] The token cache is being invalidate because project bff1175c6a224f239ac05b25cbe9e48a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:25:56.271896 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-545ce705-a02e-4ad2-be45-8b1d752554a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:56.272540 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-545ce705-a02e-4ad2-be45-8b1d752554a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:25:56.274431 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1447/2898] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:25:56 2022] DELETE /identity/v3/projects/bff1175c6a224f239ac05b25cbe9e48a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:25:56.279248 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945708f-fc50-4474-b4cd-98b6f9cc4704 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:56.279526 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945708f-fc50-4474-b4cd-98b6f9cc4704 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:56.279794 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945708f-fc50-4474-b4cd-98b6f9cc4704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:56.302092 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-f945708f-fc50-4474-b4cd-98b6f9cc4704 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:56.304466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-f945708f-fc50-4474-b4cd-98b6f9cc4704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:56.381862 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1452/2899] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:56.391356 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9e9e4-142f-4de5-a921-d889778cdf4a None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:56.391642 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9e9e4-142f-4de5-a921-d889778cdf4a None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:56.391922 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9e9e4-142f-4de5-a921-d889778cdf4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:56.414879 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-ded9e9e4-142f-4de5-a921-d889778cdf4a None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:56.417149 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-ded9e9e4-142f-4de5-a921-d889778cdf4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:56.427605 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1448/2900] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:56.432458 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945708f-fc50-4474-b4cd-98b6f9cc4704 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:56.449101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945708f-fc50-4474-b4cd-98b6f9cc4704 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:56.454851 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38a356-614a-4937-b585-22b654e513b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:56.455597 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38a356-614a-4937-b585-22b654e513b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:56.455953 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38a356-614a-4937-b585-22b654e513b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:56.456303 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38a356-614a-4937-b585-22b654e513b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:56.457052 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38a356-614a-4937-b585-22b654e513b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81782) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:25:56.457432 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38a356-614a-4937-b585-22b654e513b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:56.459988 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38a356-614a-4937-b585-22b654e513b5 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:56.473954 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1453/2901] 10.222.0.73 () {58 vars in 1153 bytes} [Mon Aug 29 20:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:25:56.478599 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded9e9e4-142f-4de5-a921-d889778cdf4a None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:56.489162 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded9e9e4-142f-4de5-a921-d889778cdf4a None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:56.494291 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e634335-2432-4979-bef1-b1b6196585cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:56.494916 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e634335-2432-4979-bef1-b1b6196585cb admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:56.495192 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e634335-2432-4979-bef1-b1b6196585cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:56.495461 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e634335-2432-4979-bef1-b1b6196585cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:56.496113 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e634335-2432-4979-bef1-b1b6196585cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:56.498273 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e634335-2432-4979-bef1-b1b6196585cb admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:56.537585 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1449/2902] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:25:56 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:25:56.538871 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38a356-614a-4937-b585-22b654e513b5 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:56.549535 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38a356-614a-4937-b585-22b654e513b5 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:56.609626 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9b1eb-9124-403e-844b-eadd4bb86d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:56.610366 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9b1eb-9124-403e-844b-eadd4bb86d91 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:56.610718 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9b1eb-9124-403e-844b-eadd4bb86d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:56.611856 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9b1eb-9124-403e-844b-eadd4bb86d91 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:56.612566 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a9b1eb-9124-403e-844b-eadd4bb86d91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:56.614649 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a9b1eb-9124-403e-844b-eadd4bb86d91 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:56.638141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1454/2903] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:25:56 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:25:56.642298 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e634335-2432-4979-bef1-b1b6196585cb admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:56.654056 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e634335-2432-4979-bef1-b1b6196585cb admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:56.658890 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf0f8d-ab15-47f4-bfe7-30748480fb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:56.659497 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf0f8d-ab15-47f4-bfe7-30748480fb7a admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:56.659777 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf0f8d-ab15-47f4-bfe7-30748480fb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:56.660046 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf0f8d-ab15-47f4-bfe7-30748480fb7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:56.660689 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf0f8d-ab15-47f4-bfe7-30748480fb7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:56.662774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf0f8d-ab15-47f4-bfe7-30748480fb7a admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:56.673246 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1450/2904] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:25:56.677644 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9b1eb-9124-403e-844b-eadd4bb86d91 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:56.688022 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9b1eb-9124-403e-844b-eadd4bb86d91 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:56.693164 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294a0c7-520d-4af2-ba47-dc0bf0d631db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:56.693797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294a0c7-520d-4af2-ba47-dc0bf0d631db admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:56.694078 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294a0c7-520d-4af2-ba47-dc0bf0d631db admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:56.694347 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294a0c7-520d-4af2-ba47-dc0bf0d631db admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:56.694988 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9294a0c7-520d-4af2-ba47-dc0bf0d631db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:56.696612 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9294a0c7-520d-4af2-ba47-dc0bf0d631db admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:56.707559 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1455/2905] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:25:56.712961 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf0f8d-ab15-47f4-bfe7-30748480fb7a admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:56.723524 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf0f8d-ab15-47f4-bfe7-30748480fb7a admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:56.728367 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf29ce-83fa-4657-b9f6-8c9630bdf033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:56.729052 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf29ce-83fa-4657-b9f6-8c9630bdf033 admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:56.729330 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf29ce-83fa-4657-b9f6-8c9630bdf033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:56.729612 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf29ce-83fa-4657-b9f6-8c9630bdf033 admin admin] PATH_INFO: `/v3/projects/e7d18c1395a745c6a2795560b2b01b3f/users/5ed184e9e4cb476089ba1bfcb4818600/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:56.746906 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf29ce-83fa-4657-b9f6-8c9630bdf033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7d18c1395a745c6a2795560b2b01b3f, user_id=5ed184e9e4cb476089ba1bfcb4818600, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:56.749630 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf29ce-83fa-4657-b9f6-8c9630bdf033 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:56.764094 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1451/2906] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:25:56 2022] PUT /identity/v3/projects/e7d18c1395a745c6a2795560b2b01b3f/users/5ed184e9e4cb476089ba1bfcb4818600/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:56.775307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca98ca6-8e16-4af7-8296-e4c4412991d9 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:56.775882 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca98ca6-8e16-4af7-8296-e4c4412991d9 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:56.775882 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca98ca6-8e16-4af7-8296-e4c4412991d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:56.806092 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-dca98ca6-8e16-4af7-8296-e4c4412991d9 None None] MFA Rules not processed for user `5ed184e9e4cb476089ba1bfcb4818600`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:56.807750 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-dca98ca6-8e16-4af7-8296-e4c4412991d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:56.865399 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:56.889206 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1456/2907] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:25:56.893846 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf29ce-83fa-4657-b9f6-8c9630bdf033 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:56.969605 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf29ce-83fa-4657-b9f6-8c9630bdf033 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:56.974510 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3d5f9-5bf4-4bbe-bfb8-8b98106c80e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:56.975119 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3d5f9-5bf4-4bbe-bfb8-8b98106c80e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:56.975398 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3d5f9-5bf4-4bbe-bfb8-8b98106c80e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:56.975672 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3d5f9-5bf4-4bbe-bfb8-8b98106c80e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:56.976321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3d5f9-5bf4-4bbe-bfb8-8b98106c80e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:56.978858 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3d5f9-5bf4-4bbe-bfb8-8b98106c80e1 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:57.011562 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1452/2908] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:25:56 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:25:57.015748 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca98ca6-8e16-4af7-8296-e4c4412991d9 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:57.027743 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca98ca6-8e16-4af7-8296-e4c4412991d9 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:57.095415 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f953a-8bed-4af7-8299-6c686b9424c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:57.096119 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f953a-8bed-4af7-8299-6c686b9424c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:57.096320 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f953a-8bed-4af7-8299-6c686b9424c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:57.096588 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f953a-8bed-4af7-8299-6c686b9424c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:57.097266 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f953a-8bed-4af7-8299-6c686b9424c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:57.100079 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f953a-8bed-4af7-8299-6c686b9424c0 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:57.124817 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1457/2909] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:25:57 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:25:57.129359 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3d5f9-5bf4-4bbe-bfb8-8b98106c80e1 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:57.147585 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3d5f9-5bf4-4bbe-bfb8-8b98106c80e1 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:57.152746 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c22a948-d2b5-41fb-b20c-68f902a141f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:57.153366 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22a948-d2b5-41fb-b20c-68f902a141f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:57.153646 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22a948-d2b5-41fb-b20c-68f902a141f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:57.153951 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22a948-d2b5-41fb-b20c-68f902a141f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:57.154603 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c22a948-d2b5-41fb-b20c-68f902a141f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:57.156211 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c22a948-d2b5-41fb-b20c-68f902a141f7 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:57.167943 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1453/2910] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:25:57.172446 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f953a-8bed-4af7-8299-6c686b9424c0 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:57.183133 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f953a-8bed-4af7-8299-6c686b9424c0 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:57.188791 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0624c-afc7-4441-a436-63bab56bf157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:57.189404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0624c-afc7-4441-a436-63bab56bf157 admin admin] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:57.189687 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0624c-afc7-4441-a436-63bab56bf157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:57.189985 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0624c-afc7-4441-a436-63bab56bf157 admin admin] PATH_INFO: `/v3/roles` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:57.190632 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0624c-afc7-4441-a436-63bab56bf157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:57.192256 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0624c-afc7-4441-a436-63bab56bf157 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:57.204487 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1458/2911] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:25:57.211313 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c22a948-d2b5-41fb-b20c-68f902a141f7 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:57.224316 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c22a948-d2b5-41fb-b20c-68f902a141f7 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:57.229517 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a91a49-704e-4042-8721-2cd86894760a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:57.230229 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a91a49-704e-4042-8721-2cd86894760a admin admin] REQUEST_METHOD: `PUT` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:57.230506 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a91a49-704e-4042-8721-2cd86894760a admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:57.230782 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a91a49-704e-4042-8721-2cd86894760a admin admin] PATH_INFO: `/v3/projects/d594af6f392b4bb7930fffdbce864e86/users/12468fab05be42a1afaf1434f3af0a7b/roles/9265e1fd617b4c0b8fbf267885394060` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:57.254488 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a91a49-704e-4042-8721-2cd86894760a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d594af6f392b4bb7930fffdbce864e86, user_id=12468fab05be42a1afaf1434f3af0a7b, role_id=9265e1fd617b4c0b8fbf267885394060)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:57.260409 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a91a49-704e-4042-8721-2cd86894760a admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:57.278875 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1454/2912] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:25:57 2022] PUT /identity/v3/projects/d594af6f392b4bb7930fffdbce864e86/users/12468fab05be42a1afaf1434f3af0a7b/roles/9265e1fd617b4c0b8fbf267885394060 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:57.292557 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8e994-13d3-43c9-8bff-7f0c23a26268 None None] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:57.293014 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8e994-13d3-43c9-8bff-7f0c23a26268 None None] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:57.293467 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8e994-13d3-43c9-8bff-7f0c23a26268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:57.333273 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.auth.core [None req-06c8e994-13d3-43c9-8bff-7f0c23a26268 None None] MFA Rules not processed for user `12468fab05be42a1afaf1434f3af0a7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81782) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:57.335097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-06c8e994-13d3-43c9-8bff-7f0c23a26268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:57.416224 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1459/2913] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:57.417542 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a91a49-704e-4042-8721-2cd86894760a admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:57.457267 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:57.487810 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a91a49-704e-4042-8721-2cd86894760a admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:57.493348 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15042adf-480e-410b-9378-3b20515f04a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:57.493996 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15042adf-480e-410b-9378-3b20515f04a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:57.494277 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15042adf-480e-410b-9378-3b20515f04a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:57.494552 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15042adf-480e-410b-9378-3b20515f04a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:57.495182 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15042adf-480e-410b-9378-3b20515f04a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:57.497072 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15042adf-480e-410b-9378-3b20515f04a3 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:57.525405 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1455/2914] 10.222.0.73 () {60 vars in 1153 bytes} [Mon Aug 29 20:25:57 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:25:57.529542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8e994-13d3-43c9-8bff-7f0c23a26268 None None] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:57.541033 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8e994-13d3-43c9-8bff-7f0c23a26268 None None] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:57.588718 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:57.598260 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b818ad8-f2c2-4043-80ad-5642691d1b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:57.598870 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b818ad8-f2c2-4043-80ad-5642691d1b7c admin admin] REQUEST_METHOD: `POST` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:57.599148 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b818ad8-f2c2-4043-80ad-5642691d1b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:57.599416 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b818ad8-f2c2-4043-80ad-5642691d1b7c admin admin] PATH_INFO: `/v3/users` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:57.600192 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b818ad8-f2c2-4043-80ad-5642691d1b7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:57.602510 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b818ad8-f2c2-4043-80ad-5642691d1b7c admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:57.624441 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1460/2915] 10.222.0.73 () {60 vars in 1144 bytes} [Mon Aug 29 20:25:57 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:25:57.628465 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15042adf-480e-410b-9378-3b20515f04a3 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:57.639510 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15042adf-480e-410b-9378-3b20515f04a3 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:57.644241 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e299175-e97d-43cd-b039-90ce9261cc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:57.644774 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e299175-e97d-43cd-b039-90ce9261cc2c admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:57.645055 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e299175-e97d-43cd-b039-90ce9261cc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:57.645326 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e299175-e97d-43cd-b039-90ce9261cc2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:57.646113 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e299175-e97d-43cd-b039-90ce9261cc2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:57.647729 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e299175-e97d-43cd-b039-90ce9261cc2c admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:57.673231 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1456/2916] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:25:57.678655 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b818ad8-f2c2-4043-80ad-5642691d1b7c admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:57.689001 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b818ad8-f2c2-4043-80ad-5642691d1b7c admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:57.694497 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443623ef-f19b-49fc-b402-0ea266f59a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:57.695152 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443623ef-f19b-49fc-b402-0ea266f59a37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:57.695450 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443623ef-f19b-49fc-b402-0ea266f59a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:57.695716 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443623ef-f19b-49fc-b402-0ea266f59a37 admin admin] PATH_INFO: `/v3/projects/2343391e104547ab9a0a45194cceed6e/users/cc8a3285e1894cc8ab426cd1cdeec0b9/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:57.711479 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443623ef-f19b-49fc-b402-0ea266f59a37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2343391e104547ab9a0a45194cceed6e, user_id=cc8a3285e1894cc8ab426cd1cdeec0b9, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:57.713450 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443623ef-f19b-49fc-b402-0ea266f59a37 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:57.726307 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1461/2917] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:25:57 2022] PUT /identity/v3/projects/2343391e104547ab9a0a45194cceed6e/users/cc8a3285e1894cc8ab426cd1cdeec0b9/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:57.730233 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e299175-e97d-43cd-b039-90ce9261cc2c admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:57.740852 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e299175-e97d-43cd-b039-90ce9261cc2c admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:57.796570 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf69ea8-1db3-47ae-9775-3ec325c3094f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:57.797212 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf69ea8-1db3-47ae-9775-3ec325c3094f admin admin] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:57.797491 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf69ea8-1db3-47ae-9775-3ec325c3094f admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:57.797877 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf69ea8-1db3-47ae-9775-3ec325c3094f admin admin] PATH_INFO: `/v3/roles` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:57.798519 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf69ea8-1db3-47ae-9775-3ec325c3094f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:57.800108 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf69ea8-1db3-47ae-9775-3ec325c3094f admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:57.810661 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1457/2918] 10.222.0.73 () {58 vars in 1122 bytes} [Mon Aug 29 20:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:25:57.814902 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443623ef-f19b-49fc-b402-0ea266f59a37 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:57.826440 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443623ef-f19b-49fc-b402-0ea266f59a37 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:57.831155 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ba4a7-72ab-4380-85ad-4dcded72a755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:57.831807 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ba4a7-72ab-4380-85ad-4dcded72a755 admin admin] REQUEST_METHOD: `PUT` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:57.832087 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ba4a7-72ab-4380-85ad-4dcded72a755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:57.832362 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ba4a7-72ab-4380-85ad-4dcded72a755 admin admin] PATH_INFO: `/v3/projects/2343391e104547ab9a0a45194cceed6e/users/cc8a3285e1894cc8ab426cd1cdeec0b9/roles/296ee2c194824485af71618619f82e09` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:57.834540 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ba4a7-72ab-4380-85ad-4dcded72a755 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2343391e104547ab9a0a45194cceed6e, user_id=cc8a3285e1894cc8ab426cd1cdeec0b9, role_id=296ee2c194824485af71618619f82e09)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:57.837178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ba4a7-72ab-4380-85ad-4dcded72a755 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:57.847681 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1462/2919] 10.222.0.73 () {60 vars in 1483 bytes} [Mon Aug 29 20:25:57 2022] PUT /identity/v3/projects/2343391e104547ab9a0a45194cceed6e/users/cc8a3285e1894cc8ab426cd1cdeec0b9/roles/296ee2c194824485af71618619f82e09 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:25:57.859567 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6514e-63c1-4b4e-8d95-5f61e42b94c2 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:57.859856 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6514e-63c1-4b4e-8d95-5f61e42b94c2 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:57.860134 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6514e-63c1-4b4e-8d95-5f61e42b94c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:57.889171 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-c5d6514e-63c1-4b4e-8d95-5f61e42b94c2 None None] MFA Rules not processed for user `cc8a3285e1894cc8ab426cd1cdeec0b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:57.890442 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c5d6514e-63c1-4b4e-8d95-5f61e42b94c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:57.961803 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1458/2920] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:25:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:57.973285 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ba4a7-72ab-4380-85ad-4dcded72a755 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:57.975772 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-e48ba4a7-72ab-4380-85ad-4dcded72a755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:58.042149 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ba4a7-72ab-4380-85ad-4dcded72a755 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:58.045949 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ffe24-fd5d-447b-b26e-92d451a52418 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:58.046558 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ffe24-fd5d-447b-b26e-92d451a52418 service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:58.046848 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ffe24-fd5d-447b-b26e-92d451a52418 service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:58.047136 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ffe24-fd5d-447b-b26e-92d451a52418 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:58.106856 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ffe24-fd5d-447b-b26e-92d451a52418 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:58.108300 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ffe24-fd5d-447b-b26e-92d451a52418 service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:58.117698 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1463/2921] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:58.377903 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6514e-63c1-4b4e-8d95-5f61e42b94c2 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:58.379342 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-c5d6514e-63c1-4b4e-8d95-5f61e42b94c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:58.413401 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:58.445397 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6514e-63c1-4b4e-8d95-5f61e42b94c2 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:58.449338 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361b006-a1e9-4e92-acaf-98e57c205edd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:58.450305 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361b006-a1e9-4e92-acaf-98e57c205edd service neutron] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:58.450534 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361b006-a1e9-4e92-acaf-98e57c205edd service neutron] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:58.450644 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361b006-a1e9-4e92-acaf-98e57c205edd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:58.452753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-8361b006-a1e9-4e92-acaf-98e57c205edd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:58.524932 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361b006-a1e9-4e92-acaf-98e57c205edd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:58.526656 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361b006-a1e9-4e92-acaf-98e57c205edd service neutron] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:58.537533 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1459/2922] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:59.688656 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ffe24-fd5d-447b-b26e-92d451a52418 service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:59.691354 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-850ffe24-fd5d-447b-b26e-92d451a52418 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:59.705347 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ffe24-fd5d-447b-b26e-92d451a52418 service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:59.716858 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23960ad6-e318-41c3-a8c5-257e44f0cc5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:59.717564 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23960ad6-e318-41c3-a8c5-257e44f0cc5e service cinder] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:59.717978 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23960ad6-e318-41c3-a8c5-257e44f0cc5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:59.718351 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23960ad6-e318-41c3-a8c5-257e44f0cc5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:59.783606 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23960ad6-e318-41c3-a8c5-257e44f0cc5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:59.785047 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23960ad6-e318-41c3-a8c5-257e44f0cc5e service cinder] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:59.789266 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:59.797619 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1464/2923] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:00.397698 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361b006-a1e9-4e92-acaf-98e57c205edd service neutron] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:00.408561 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361b006-a1e9-4e92-acaf-98e57c205edd service neutron] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:00.413293 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2ac06-9539-4a53-a30e-8f9d39024cf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0b2ca8d14a47239a26912e58348f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:00.414172 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2ac06-9539-4a53-a30e-8f9d39024cf2 service cinder] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:00.414648 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2ac06-9539-4a53-a30e-8f9d39024cf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:00.414918 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2ac06-9539-4a53-a30e-8f9d39024cf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:00.425466 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2ac06-9539-4a53-a30e-8f9d39024cf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:00.427596 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2ac06-9539-4a53-a30e-8f9d39024cf2 service cinder] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:00.438929 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1460/2924] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:04.526466 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23960ad6-e318-41c3-a8c5-257e44f0cc5e service cinder] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:04.539971 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23960ad6-e318-41c3-a8c5-257e44f0cc5e service cinder] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:04.544718 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb49623-af3a-4984-9358-86cd578efefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:04.545570 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb49623-af3a-4984-9358-86cd578efefc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:04.545879 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb49623-af3a-4984-9358-86cd578efefc admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:04.546150 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb49623-af3a-4984-9358-86cd578efefc admin admin] PATH_INFO: `/v3/users/5ed184e9e4cb476089ba1bfcb4818600` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:04.547351 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb49623-af3a-4984-9358-86cd578efefc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ed184e9e4cb476089ba1bfcb4818600)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:04.549404 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb49623-af3a-4984-9358-86cd578efefc admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:04.599718 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-7bb49623-af3a-4984-9358-86cd578efefc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ed184e9e4cb476089ba1bfcb4818600'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:04.615192 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-7bb49623-af3a-4984-9358-86cd578efefc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ed184e9e4cb476089ba1bfcb4818600'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:04.622977 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-7bb49623-af3a-4984-9358-86cd578efefc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ed184e9e4cb476089ba1bfcb4818600'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:04.630165 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-7bb49623-af3a-4984-9358-86cd578efefc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ed184e9e4cb476089ba1bfcb4818600'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:04.630863 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-7bb49623-af3a-4984-9358-86cd578efefc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ed184e9e4cb476089ba1bfcb4818600'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:04.636827 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1465/2925] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:26:04 2022] DELETE /identity/v3/users/5ed184e9e4cb476089ba1bfcb4818600 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:26:04.641582 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb65a5e-53e0-40eb-8b4c-b6c578ed58e6 None None] REQUEST_METHOD: `POST` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:04.642147 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb65a5e-53e0-40eb-8b4c-b6c578ed58e6 None None] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:04.642266 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb65a5e-53e0-40eb-8b4c-b6c578ed58e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:04.663779 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.auth.core [None req-fdb65a5e-53e0-40eb-8b4c-b6c578ed58e6 None None] MFA Rules not processed for user `7ab6b80a7eaf40e691fcd95e0949b22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81781) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:04.665154 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-fdb65a5e-53e0-40eb-8b4c-b6c578ed58e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:04.727027 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1461/2926] 10.222.0.73 () {58 vars in 958 bytes} [Mon Aug 29 20:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:04.736734 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb49623-af3a-4984-9358-86cd578efefc admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:04.738237 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-7bb49623-af3a-4984-9358-86cd578efefc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:04.814323 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb49623-af3a-4984-9358-86cd578efefc admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:04.822608 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685f438-f206-4dd4-aa5e-15efca75e1b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ba58d774674ed7b233ad6416db068a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:04.823744 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685f438-f206-4dd4-aa5e-15efca75e1b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:04.824518 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685f438-f206-4dd4-aa5e-15efca75e1b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:04.825378 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685f438-f206-4dd4-aa5e-15efca75e1b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:04.848146 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685f438-f206-4dd4-aa5e-15efca75e1b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:04.851742 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685f438-f206-4dd4-aa5e-15efca75e1b1 service neutron] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:04.873175 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1466/2927] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:05.356892 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb65a5e-53e0-40eb-8b4c-b6c578ed58e6 None None] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:05.358447 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-fdb65a5e-53e0-40eb-8b4c-b6c578ed58e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:05.379327 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb65a5e-53e0-40eb-8b4c-b6c578ed58e6 None None] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:05.389538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f984c-b7ea-4411-acb7-4a4548ab6bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:05.394990 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f984c-b7ea-4411-acb7-4a4548ab6bcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:05.395685 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f984c-b7ea-4411-acb7-4a4548ab6bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:05.396714 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f984c-b7ea-4411-acb7-4a4548ab6bcf admin admin] PATH_INFO: `/v3/projects/e7d18c1395a745c6a2795560b2b01b3f` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:05.400466 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f984c-b7ea-4411-acb7-4a4548ab6bcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7d18c1395a745c6a2795560b2b01b3f)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:05.403053 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f984c-b7ea-4411-acb7-4a4548ab6bcf admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:05.475188 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b33f984c-b7ea-4411-acb7-4a4548ab6bcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7d18c1395a745c6a2795560b2b01b3f', 'request_id': 'req-b33f984c-b7ea-4411-acb7-4a4548ab6bcf'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:05.482406 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b33f984c-b7ea-4411-acb7-4a4548ab6bcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7d18c1395a745c6a2795560b2b01b3f', 'request_id': 'req-b33f984c-b7ea-4411-acb7-4a4548ab6bcf'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:05.485328 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685f438-f206-4dd4-aa5e-15efca75e1b1 service neutron] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:05.487337 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-3685f438-f206-4dd4-aa5e-15efca75e1b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:05.488350 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:05.495586 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b33f984c-b7ea-4411-acb7-4a4548ab6bcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7d18c1395a745c6a2795560b2b01b3f', 'request_id': 'req-b33f984c-b7ea-4411-acb7-4a4548ab6bcf'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:05.514182 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b33f984c-b7ea-4411-acb7-4a4548ab6bcf admin admin] The token cache is being invalidate because project e7d18c1395a745c6a2795560b2b01b3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:26:05.514405 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b33f984c-b7ea-4411-acb7-4a4548ab6bcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:05.515069 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-b33f984c-b7ea-4411-acb7-4a4548ab6bcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:05.520860 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1462/2928] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:26:05 2022] DELETE /identity/v3/projects/e7d18c1395a745c6a2795560b2b01b3f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:26:05.525490 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f984c-b7ea-4411-acb7-4a4548ab6bcf admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:05.530495 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-b33f984c-b7ea-4411-acb7-4a4548ab6bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:05.619512 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685f438-f206-4dd4-aa5e-15efca75e1b1 service neutron] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:05.628538 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f984c-b7ea-4411-acb7-4a4548ab6bcf admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:05.633662 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:05.634726 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e4e76-9eea-41c3-ba96-9a45823489d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:05.635436 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e4e76-9eea-41c3-ba96-9a45823489d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:05.635691 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e4e76-9eea-41c3-ba96-9a45823489d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:05.635992 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e4e76-9eea-41c3-ba96-9a45823489d3 admin admin] PATH_INFO: `/v3/users/12468fab05be42a1afaf1434f3af0a7b` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:05.637163 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e4e76-9eea-41c3-ba96-9a45823489d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12468fab05be42a1afaf1434f3af0a7b)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:05.639139 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e4e76-9eea-41c3-ba96-9a45823489d3 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:05.696254 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f4b15d-a496-4b19-9a69-ce7bf387be46 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab2e318b2854c46bc7ed2151a44a2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:05.696935 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4b15d-a496-4b19-9a69-ce7bf387be46 service placement] REQUEST_METHOD: `GET` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:05.697362 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4b15d-a496-4b19-9a69-ce7bf387be46 service placement] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:05.697861 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4b15d-a496-4b19-9a69-ce7bf387be46 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:05.698713 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9a5e4e76-9eea-41c3-ba96-9a45823489d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12468fab05be42a1afaf1434f3af0a7b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:05.706232 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-35f4b15d-a496-4b19-9a69-ce7bf387be46 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:05.711791 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9a5e4e76-9eea-41c3-ba96-9a45823489d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12468fab05be42a1afaf1434f3af0a7b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:05.720661 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f4b15d-a496-4b19-9a69-ce7bf387be46 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:05.722553 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f4b15d-a496-4b19-9a69-ce7bf387be46 service placement] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:05.726904 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9a5e4e76-9eea-41c3-ba96-9a45823489d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12468fab05be42a1afaf1434f3af0a7b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:05.734673 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1467/2929] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:26:05.735542 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f4b15d-a496-4b19-9a69-ce7bf387be46 service placement] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:05.736847 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-35f4b15d-a496-4b19-9a69-ce7bf387be46 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:05.753746 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9a5e4e76-9eea-41c3-ba96-9a45823489d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12468fab05be42a1afaf1434f3af0a7b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:05.755417 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-9a5e4e76-9eea-41c3-ba96-9a45823489d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12468fab05be42a1afaf1434f3af0a7b'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:05.760121 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f4b15d-a496-4b19-9a69-ce7bf387be46 service placement] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:05.765589 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d0efd-e4da-4095-bb08-6066f441a240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:05.766420 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d0efd-e4da-4095-bb08-6066f441a240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:05.766700 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d0efd-e4da-4095-bb08-6066f441a240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:05.767042 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d0efd-e4da-4095-bb08-6066f441a240 admin admin] PATH_INFO: `/v3/users/8640600ca0d745cda547a1027c947089` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:05.768327 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d0efd-e4da-4095-bb08-6066f441a240 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8640600ca0d745cda547a1027c947089)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:05.772156 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d0efd-e4da-4095-bb08-6066f441a240 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:05.777730 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1463/2930] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:26:05 2022] DELETE /identity/v3/users/12468fab05be42a1afaf1434f3af0a7b => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:26:05.830352 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e88d0efd-e4da-4095-bb08-6066f441a240 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8640600ca0d745cda547a1027c947089'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:05.847784 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e88d0efd-e4da-4095-bb08-6066f441a240 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8640600ca0d745cda547a1027c947089'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:05.861697 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e88d0efd-e4da-4095-bb08-6066f441a240 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8640600ca0d745cda547a1027c947089'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:05.871115 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e88d0efd-e4da-4095-bb08-6066f441a240 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8640600ca0d745cda547a1027c947089'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:05.871855 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-e88d0efd-e4da-4095-bb08-6066f441a240 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8640600ca0d745cda547a1027c947089'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:05.879067 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1468/2931] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:26:05 2022] DELETE /identity/v3/users/8640600ca0d745cda547a1027c947089 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:26:06.109509 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e4e76-9eea-41c3-ba96-9a45823489d3 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:06.111144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-9a5e4e76-9eea-41c3-ba96-9a45823489d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:06.117135 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d0efd-e4da-4095-bb08-6066f441a240 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:06.118956 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-e88d0efd-e4da-4095-bb08-6066f441a240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:06.194430 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e4e76-9eea-41c3-ba96-9a45823489d3 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:06.208718 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:06.216055 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d0efd-e4da-4095-bb08-6066f441a240 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:06.221539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6fe1a-12c7-417a-bba2-70e56b58589c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:06.222225 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6fe1a-12c7-417a-bba2-70e56b58589c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:06.222505 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6fe1a-12c7-417a-bba2-70e56b58589c admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:06.222768 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6fe1a-12c7-417a-bba2-70e56b58589c admin admin] PATH_INFO: `/v3/projects/d594af6f392b4bb7930fffdbce864e86` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:06.225338 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6fe1a-12c7-417a-bba2-70e56b58589c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d594af6f392b4bb7930fffdbce864e86)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:06.227797 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6fe1a-12c7-417a-bba2-70e56b58589c admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:06.250403 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:06.260157 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d72bd9-e364-4ff5-9733-7f648e2978b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:06.260802 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d72bd9-e364-4ff5-9733-7f648e2978b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:06.261080 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d72bd9-e364-4ff5-9733-7f648e2978b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:06.261358 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d72bd9-e364-4ff5-9733-7f648e2978b1 admin admin] PATH_INFO: `/v3/projects/ff3969ffe8ff411b8773e1f3f3cb0b40` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:06.262592 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d72bd9-e364-4ff5-9733-7f648e2978b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff3969ffe8ff411b8773e1f3f3cb0b40)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:06.264492 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d72bd9-e364-4ff5-9733-7f648e2978b1 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:06.280235 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1ed6fe1a-12c7-417a-bba2-70e56b58589c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd594af6f392b4bb7930fffdbce864e86', 'request_id': 'req-1ed6fe1a-12c7-417a-bba2-70e56b58589c'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:06.286783 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1ed6fe1a-12c7-417a-bba2-70e56b58589c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd594af6f392b4bb7930fffdbce864e86', 'request_id': 'req-1ed6fe1a-12c7-417a-bba2-70e56b58589c'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:06.295244 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1ed6fe1a-12c7-417a-bba2-70e56b58589c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd594af6f392b4bb7930fffdbce864e86', 'request_id': 'req-1ed6fe1a-12c7-417a-bba2-70e56b58589c'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:06.307181 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1ed6fe1a-12c7-417a-bba2-70e56b58589c admin admin] The token cache is being invalidate because project d594af6f392b4bb7930fffdbce864e86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:26:06.307514 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1ed6fe1a-12c7-417a-bba2-70e56b58589c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:06.308159 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-1ed6fe1a-12c7-417a-bba2-70e56b58589c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:06.310123 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1469/2932] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:26:06 2022] DELETE /identity/v3/projects/d594af6f392b4bb7930fffdbce864e86 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:26:06.314168 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6fe1a-12c7-417a-bba2-70e56b58589c admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:06.315700 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-61d72bd9-e364-4ff5-9733-7f648e2978b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff3969ffe8ff411b8773e1f3f3cb0b40', 'request_id': 'req-61d72bd9-e364-4ff5-9733-7f648e2978b1'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:06.317030 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-1ed6fe1a-12c7-417a-bba2-70e56b58589c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:06.322437 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-61d72bd9-e364-4ff5-9733-7f648e2978b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff3969ffe8ff411b8773e1f3f3cb0b40', 'request_id': 'req-61d72bd9-e364-4ff5-9733-7f648e2978b1'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:06.329913 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-61d72bd9-e364-4ff5-9733-7f648e2978b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff3969ffe8ff411b8773e1f3f3cb0b40', 'request_id': 'req-61d72bd9-e364-4ff5-9733-7f648e2978b1'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:06.350438 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-61d72bd9-e364-4ff5-9733-7f648e2978b1 admin admin] The token cache is being invalidate because project ff3969ffe8ff411b8773e1f3f3cb0b40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:26:06.350732 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-61d72bd9-e364-4ff5-9733-7f648e2978b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:06.351386 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-61d72bd9-e364-4ff5-9733-7f648e2978b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:06.352897 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1464/2933] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:26:06 2022] DELETE /identity/v3/projects/ff3969ffe8ff411b8773e1f3f3cb0b40 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:26:06.356822 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d72bd9-e364-4ff5-9733-7f648e2978b1 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:06.358203 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-61d72bd9-e364-4ff5-9733-7f648e2978b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:06.391604 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6fe1a-12c7-417a-bba2-70e56b58589c admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:06.426684 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d72bd9-e364-4ff5-9733-7f648e2978b1 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:06.431675 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7a061-5752-4543-94c4-54d1a276c022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:06.432321 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7a061-5752-4543-94c4-54d1a276c022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:06.432598 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7a061-5752-4543-94c4-54d1a276c022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:06.432862 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7a061-5752-4543-94c4-54d1a276c022 admin admin] PATH_INFO: `/v3/users/a73812ad386243e29fc41ef97037ae85` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:06.434085 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e7a061-5752-4543-94c4-54d1a276c022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a73812ad386243e29fc41ef97037ae85)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:06.436032 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e7a061-5752-4543-94c4-54d1a276c022 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:06.451888 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d34d89-3e86-4357-a68e-08a66523df39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:06.452539 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d34d89-3e86-4357-a68e-08a66523df39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:06.452823 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d34d89-3e86-4357-a68e-08a66523df39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:06.453101 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d34d89-3e86-4357-a68e-08a66523df39 admin admin] PATH_INFO: `/v3/users/cc8a3285e1894cc8ab426cd1cdeec0b9` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:06.454345 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d34d89-3e86-4357-a68e-08a66523df39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc8a3285e1894cc8ab426cd1cdeec0b9)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:06.456280 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d34d89-3e86-4357-a68e-08a66523df39 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:06.488015 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-38e7a061-5752-4543-94c4-54d1a276c022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a73812ad386243e29fc41ef97037ae85'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:06.490270 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:06.497233 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-38e7a061-5752-4543-94c4-54d1a276c022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a73812ad386243e29fc41ef97037ae85'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:06.508839 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-30d34d89-3e86-4357-a68e-08a66523df39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc8a3285e1894cc8ab426cd1cdeec0b9'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:06.511536 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-38e7a061-5752-4543-94c4-54d1a276c022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a73812ad386243e29fc41ef97037ae85'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:06.516232 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:06.519406 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-38e7a061-5752-4543-94c4-54d1a276c022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a73812ad386243e29fc41ef97037ae85'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:06.521093 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-38e7a061-5752-4543-94c4-54d1a276c022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a73812ad386243e29fc41ef97037ae85'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:06.525457 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-30d34d89-3e86-4357-a68e-08a66523df39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc8a3285e1894cc8ab426cd1cdeec0b9'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:06.527844 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1465/2934] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:26:06 2022] DELETE /identity/v3/users/a73812ad386243e29fc41ef97037ae85 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:26:06.535141 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-30d34d89-3e86-4357-a68e-08a66523df39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc8a3285e1894cc8ab426cd1cdeec0b9'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:06.542996 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-30d34d89-3e86-4357-a68e-08a66523df39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc8a3285e1894cc8ab426cd1cdeec0b9'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:06.543674 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-30d34d89-3e86-4357-a68e-08a66523df39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc8a3285e1894cc8ab426cd1cdeec0b9'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:06.549538 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1470/2935] 10.222.0.73 () {58 vars in 1224 bytes} [Mon Aug 29 20:26:06 2022] DELETE /identity/v3/users/cc8a3285e1894cc8ab426cd1cdeec0b9 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:26:06.730575 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7a061-5752-4543-94c4-54d1a276c022 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:06.732107 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-38e7a061-5752-4543-94c4-54d1a276c022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:06.825638 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7a061-5752-4543-94c4-54d1a276c022 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:06.861429 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d34d89-3e86-4357-a68e-08a66523df39 admin admin] Authenticating user token {{(pid=81782) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:06.863097 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.fernet_utils [None req-30d34d89-3e86-4357-a68e-08a66523df39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81782) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:06.922783 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4631b-b7fb-49f0-b9ac-5571bac048c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:06.923440 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4631b-b7fb-49f0-b9ac-5571bac048c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:06.923725 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4631b-b7fb-49f0-b9ac-5571bac048c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:06.924001 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4631b-b7fb-49f0-b9ac-5571bac048c3 admin admin] PATH_INFO: `/v3/projects/a70786727b784db6972c1f3f1433dc36` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:06.925566 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be4631b-b7fb-49f0-b9ac-5571bac048c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a70786727b784db6972c1f3f1433dc36)` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:06.928138 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be4631b-b7fb-49f0-b9ac-5571bac048c3 admin admin] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:06.951003 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d34d89-3e86-4357-a68e-08a66523df39 admin admin] Validating token access rules against request {{(pid=81782) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:06.955258 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8563358-23bf-4883-b041-ada2c83516c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab6b80a7eaf40e691fcd95e0949b22e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee34b4bd3d8043258a1a5d3588fa8f9c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81782) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:06.955900 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8563358-23bf-4883-b041-ada2c83516c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:06.956178 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8563358-23bf-4883-b041-ada2c83516c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:06.956653 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8563358-23bf-4883-b041-ada2c83516c8 admin admin] PATH_INFO: `/v3/projects/2343391e104547ab9a0a45194cceed6e` {{(pid=81782) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:06.957898 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8563358-23bf-4883-b041-ada2c83516c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2343391e104547ab9a0a45194cceed6e)` {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:06.960138 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8563358-23bf-4883-b041-ada2c83516c8 admin admin] RBAC: Authorization granted {{(pid=81782) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:06.977586 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8be4631b-b7fb-49f0-b9ac-5571bac048c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a70786727b784db6972c1f3f1433dc36', 'request_id': 'req-8be4631b-b7fb-49f0-b9ac-5571bac048c3'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:06.983997 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8be4631b-b7fb-49f0-b9ac-5571bac048c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a70786727b784db6972c1f3f1433dc36', 'request_id': 'req-8be4631b-b7fb-49f0-b9ac-5571bac048c3'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:06.990672 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8be4631b-b7fb-49f0-b9ac-5571bac048c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a70786727b784db6972c1f3f1433dc36', 'request_id': 'req-8be4631b-b7fb-49f0-b9ac-5571bac048c3'} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:07.003346 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8be4631b-b7fb-49f0-b9ac-5571bac048c3 admin admin] The token cache is being invalidate because project a70786727b784db6972c1f3f1433dc36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81781) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:26:07.003753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8be4631b-b7fb-49f0-b9ac-5571bac048c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:07.005017 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.notifications [None req-8be4631b-b7fb-49f0-b9ac-5571bac048c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81781) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:07.006926 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1466/2936] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:26:06 2022] DELETE /identity/v3/projects/a70786727b784db6972c1f3f1433dc36 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:26:07.010591 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a8563358-23bf-4883-b041-ada2c83516c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2343391e104547ab9a0a45194cceed6e', 'request_id': 'req-a8563358-23bf-4883-b041-ada2c83516c8'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:07.017664 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a8563358-23bf-4883-b041-ada2c83516c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2343391e104547ab9a0a45194cceed6e', 'request_id': 'req-a8563358-23bf-4883-b041-ada2c83516c8'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:07.027658 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a8563358-23bf-4883-b041-ada2c83516c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2343391e104547ab9a0a45194cceed6e', 'request_id': 'req-a8563358-23bf-4883-b041-ada2c83516c8'} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:07.041084 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a8563358-23bf-4883-b041-ada2c83516c8 admin admin] The token cache is being invalidate because project 2343391e104547ab9a0a45194cceed6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81782) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:26:07.041481 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a8563358-23bf-4883-b041-ada2c83516c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:07.042439 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG keystone.notifications [None req-a8563358-23bf-4883-b041-ada2c83516c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81782) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:26:07.044767 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: [pid: 81782|app: 0|req: 1471/2937] 10.222.0.73 () {58 vars in 1233 bytes} [Mon Aug 29 20:26:06 2022] DELETE /identity/v3/projects/2343391e104547ab9a0a45194cceed6e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:26:17.000162 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=87 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:17.037438 dv-cinder-854261-2-iscsi devstack@keystone.service[81782]: DEBUG dbcounter [-] [81782] Writing DB stats keystone:SELECT=60,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81782) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:30:47.131329 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4631b-b7fb-49f0-b9ac-5571bac048c3 admin admin] Authenticating user token {{(pid=81781) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:47.132753 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-8be4631b-b7fb-49f0-b9ac-5571bac048c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:47.307985 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4631b-b7fb-49f0-b9ac-5571bac048c3 admin admin] Validating token access rules against request {{(pid=81781) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:47.381230 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264969e-e4b8-46e3-bc87-5a907bfd8792 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab2e318b2854c46bc7ed2151a44a2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f5ec615d98548159487c97cead336a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81781) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:47.381813 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264969e-e4b8-46e3-bc87-5a907bfd8792 service placement] REQUEST_METHOD: `GET` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:47.382110 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264969e-e4b8-46e3-bc87-5a907bfd8792 service placement] SCRIPT_NAME: `/identity` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:47.382384 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264969e-e4b8-46e3-bc87-5a907bfd8792 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81781) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:47.384210 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.fernet_utils [None req-1264969e-e4b8-46e3-bc87-5a907bfd8792 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81781) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:47.397351 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264969e-e4b8-46e3-bc87-5a907bfd8792 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:47.399006 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264969e-e4b8-46e3-bc87-5a907bfd8792 service placement] RBAC: Authorization granted {{(pid=81781) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:47.411144 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: [pid: 81781|app: 0|req: 1467/2938] 10.222.0.73 () {60 vars in 1365 bytes} [Mon Aug 29 20:30:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:30:57.405622 dv-cinder-854261-2-iscsi devstack@keystone.service[81781]: DEBUG dbcounter [-] [81781] Writing DB stats keystone:SELECT=89 {{(pid=81781) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}